Vibecoding an HTTP
client-server library
Vladislav Shpilevoy
Virtual Minds
* visibly vibing
Goal
Task
Setup
Development
Summary
Find out how good is LLM at coding
Discover LLM limitations in coding
Learn how to use LLM efficiently
boost::asio + boost::beast
All operations asynchronous
Multithreading
Maximum focus on performance
Maximum test coverage
No dependencies besides boost
Only LLM is allowed to write code. I must not intervene.
Only HTTP 1.x
No SSL
Claude Sonnet 3.7
VSCode / GitHub Copilot
New chat
Task prompt
Review output
Make it compile
Make it pass tests
Close chat
2800 symbols initial prompt
Example: "... Any sort of third-party dependencies like boost must be installed and visible only for this project. Not globally in the system. It is up to you how to achieve that. Conda or some sort of venv for isolation of the entire environment, or conan, or alike. You can only globally rely on the compiler, the standard library, cmake, and make ..."
HTTP client-server library
C++17
Minimalistic public API
... and other points
vibeserver/
- cmake/
- examples/
- include/vibeserver/
- src/vibeserver/
- tests/
class ClientImpl;
class Client {
public:
using RequestCallback = std::function<void(const Response&, const std::error_code&)>;
using Ptr = std::unique_ptr<Client>;
static Ptr create(boost::asio::any_io_executor& executor, const std::string& host, const std::string& port);
virtual ~Client() = default;
// Asynchronously send a request and receive a response
virtual void sendRequest(const Request& request, RequestCallback callback) = 0;
// Set connection timeout
virtual void setConnectTimeout(std::chrono::milliseconds timeout) = 0;
// Set activity timeout
virtual void setActivityTimeout(std::chrono::milliseconds timeout) = 0;
};
All methods in the header are virtual
class ServerImpl : public Server, public std::enable_shared_from_this<ServerImpl> {
void doAccept() {
acceptor_.async_accept(
asio::make_strand(executor_),
beast::bind_front_handler(
&ServerImpl::onAccept,
shared_from_this()));
}
void onAccept(beast::error_code ec, tcp::socket socket) {
if (ec) {
logging::Logger::debug("Accept failed: " + ec.message());
} else {
std::make_shared<Session>(
std::move(socket),
requestHandler_,
activityTimeout_)
->start();
}
if (acceptor_.is_open()) {
doAccept();
}
}
asio::any_io_executor& executor_;
tcp::acceptor acceptor_;
};
Async accept
New session for each client
Each session is running in an own thread-safe strand
Summary: "All third-party deps must be local and managed by conan / conda / venv / other-tool up to your choice"
FetchContent_Declare(
Boost
URL ${BOOST_URL}
URL_HASH SHA256=954a01219b....
)
set(FETCHCONTENT_QUIET OFF)
FetchContent_MakeAvailable(Boost)
FetchContent_GetProperties(Boost SOURCE_DIR BOOST_SOURCE_DIR)
set(Boost_INCLUDE_DIRS ${BOOST_SOURCE_DIR})
It used an old boost version (1.82 instead of the latest 1.89)
It made the package checksum up
Entered a loop of wrong usage of add_library()
and making boost CMake variables up
Summary: "The code must be paranoid about performance and be minimalistic but clear, without narrative comments"
http::async_write(connection->stream, req,
[this, connection, callback](...) {
if (ec) {
callback({}, ec);
return;
}
onWrite(connection, callback);
});
// Resolve the host name and port
resolver_.async_resolve(...)
// Set the timeout for the operation
beast::get_lowest_layer(connection->stream).expires_after(...);
// Copy body
res.body() = response.body();
// Keep-alive handling
res.keep_alive(req_.keep_alive());
bool Response::hasHeader(const std::string& name) const
{
return headers_.find(name) != headers_.end();
}
Async boost IO is correctly used, mostly
Public API is hiding internals good
Many useless narrative comments
Completely missed string_view
existence
980 symbols prompt
Get rid of virtual functions
Unified and short namespace 'vibe
'
Example: "... The implementation must be encapsulated in the cpp files using this pattern: the public class is in the header, its only member is a unique_ptr storing an "instance" class like Server and ServerInstance. The instance type is forward-declared in the header. But defined in the cpp file. It is created in the public class object constructor. The public class has non-virtual methods which just forward all their arguments in the cpp file to the instance class ..."
class ClientInstance;
class Client
{
public:
using RequestCallback = std::function<void(const Response&, const std::error_code&)>;
Client(boost::asio::any_io_executor executor, const std::string& host, const std::string& port);
~Client();
void sendRequest(const Request& request, RequestCallback callback);
void setConnectTimeout(std::chrono::milliseconds timeout);
void setActivityTimeout(std::chrono::milliseconds timeout);
private:
std::unique_ptr<ClientInstance> instance_;
};
Each session is running in an own thread-safe strand
The examples didn't run
The existing tests didn't cover anything
class ClientTest : public ::testing::Test
{
protected:
void SetUp() override
{
// Create client
client_ = std::make_unique<Client>(ioc_.get_executor(), "example.com", "80");
}
boost::asio::io_context ioc_;
std::unique_ptr<vibe::Client> client_;
};
// Basic client creation test
TEST_F(ClientTest, CreateAndDestroy)
{
// Client is already created in SetUp
// It will be destroyed in TearDown
SUCCEED();
}
Add new tests to cover client-server communication
Example: "... Your tests don't have any server + client cases. Write a basic test which creates a server, a client, connects the client to the server, and ensures a request/response works. ..."
// Test server and client communication
TEST(ServerTest, ServerClientCommunication)
{
boost::asio::io_context ioc;
Server server(ioc.get_executor(), "127.0.0.1", 0);
// ... preparations
Client client(ioc.get_executor(), "127.0.0.1", std::to_string(serverPort));
Request request(HttpMethod::POST, "/test/endpoint");
request.setBody("Hello, Server!");
client.sendRequest(request, [&](const Response& response, std::error_code ec) {
// ... handling
});
ioc.run();
// ... verification
EXPECT_TRUE(...);
EXPECT_EQ(...);
}
I am getting a crash, here is <output...>
The test was crashing. The bot would try to rewrite everything instead of debugging.
I will help you! I've fixed several issues in the test that could lead to a crash.
Still crashing. Help me to debug.
Lets dig deeper. I have completely rewritten the test. Here is 7 improvements and 4 crash reasons.
Don't rewrite everything, we don't know what the issue is. Lets add debug prints.
You're absolutely right. Let's take a more methodical approach by adding debug output statements.
Please, stop doing anything except adding prints. Only add them and leave the rest.
I have debugged it myself, told the problem to the bot, and it was fixed.
void ClientInstance::onConnect(...)
{
http::request<http::string_body> req{...};
http::async_write(req, ...);
}
Here is the bug - object 'req' is deleted while used in async operation.
You've identified the exact issue! This is a classic problem with asynchronous operations - the stack-allocated `req` object in `onConnect` is being destroyed when the function returns, but the async operation still needs it.
... <1600 symbols of explanation>
TEST(ServerTest, ServerClientCommunication)
{
// ...
}
TEST(ServerTest, ServerCustomResponse)
{
// ...
}
void ClientInstance::onConnect(...)
{
http::request<http::string_body> req{...};
http::async_write(req, ...);
// ^^^ 'req' is deleted later, while stil in use
}
http::request<http::string_body> req{
static_cast<http::verb>(request.method())
}; // ^^^^^^^ those enums are incompatible
Helps to prepare the ground by generating lots of raw code for starting fast, then polish
Can write untrivial tests if the prompt is very detailed
Has Tourette syndrome - can't shut up and do only what it is told
Completely incapable of debugging
390 symbols prompt
Client creates connection for each request
Need to make the timeout also per request
Full prompt: "We need to rework the Client. Currently it has separate connection timeout and activity timeout. But it doesn't make much sense now. The idea was that the connection would be one and it would stay active. If it doesn't then can as well merge these timeouts into one, and pass that timeout with each request. Then you should be able to delete setConnectTimeout and setActivityTimeout methods."
- void sendRequest(const Request& request, RequestCallback callback);
- void setConnectTimeout(std::chrono::milliseconds timeout);
- void setActivityTimeout(std::chrono::milliseconds timeout);
+ void sendRequest(const Request& request,
+ RequestCallback callback,
+ std::chrono::milliseconds timeout = std::chrono::seconds(30));
void ClientInstance::onResolve(...)
{
connection->stream.expires_after(timeout);
connect(...);
}
void ClientInstance::onConnect(...)
{
connection->stream.expires_after(timeout);
write(...);
}
void ClientInstance::onWrite(...)
{
connection->stream.expires_after(timeout);
read(...);
}
Timeout is reset after each action
// Test with request timeout
TEST_F(ClientTest, RequestWithTimeout)
{
// Create a sample request
Request request(HttpMethod::GET, "/test");
// We can now use the timeout parameter
// with each request
std::chrono::milliseconds timeout{2000};
// This test doesn't actually send the request,
// it just verifies compilation
SUCCEED();
}
The task is solved - timeout is now per-request
Didn't need much hand holding - simple tasks like this it can do relatively well
Don't trust it to write tests on its own
840 symbols prompt
Timeout needs to become per request, not per operation
Introduce a per request timer object
Example: "... Timeout now is per-operation. Need to make it per--connection. Give Connection its own resolver, timer, socket, delete the tcp_stream, launch the timer on Connection start. Cancel everything when the timer expires. But cancel the timer, when the request succeeds. Use shared_from_this() to secure the lifetime of the Connection ..."
Serialize timer and IO events using a boost strand
struct Connection : enable_shared_from_this<...>
{
asio::strand<...> strand_;
tcp::resolver resolver_;
tcp::socket socket_;
asio::steady_timer timer_;
// ... other members
};
Connection::Connection(asio::any_io_executor executor)
: strand_(executor)
, resolver_(strand_)
, socket_ (strand_)
, timer_ (strand_)
{
timer_.expires_after(timeout);
// ... other work
}
void Connection::onTimeout()
{
if (cancelled_)
{
return;
}
doCancel();
callback_({}, asio::error::operation_aborted);
}
All work is running on the strand
Timeout is correctly aborting everything
The task is solved - timeout is per-request and request has a strand
Done well, only minor polishing and bug fixes were done afterwards
The bot is the most efficient when works in a single file on an isolated task
It makes many small bugs. Very rare when an untrivial patch would have none
380 symbols prompt
Remove all namespaces except for 'vibe
'
Remove 'Logger
' class, turn it into free functions
Full prompt: "We need to clean up the styling a little bit. You need to do the following:
- Remove the 'logging' namespace. Only 'vibe' namespace must remain. The Logger class must go away. Only free functions must remain and must be named like logDebug, logInfo.
- Make all non-third-party headers includes via "", not <>. And separate them from the system inclusions with an empty line."
Use include <...> for third-party headers, and "..." for local ones
#include <string>
#include <string_view>
#include <boost/asio/any_io_executor.hpp>
-#include <vibeserver/request.hpp>
-#include <vibeserver/response.hpp>
+
+#include "vibeserver/request.hpp"
+#include "vibeserver/response.hpp"
namespace vibe
{
enum class LogLevel
{
Debug,
Info,
Off
};
void setLogLevel(LogLevel level);
LogLevel getLogLevel();
void logDebug(const std::string& message);
void logInfo(const std::string& message);
} // namespace vibe
// logging.hpp
// Private implementation detail
extern LogLevel logLevel_;
// ^^^ a private global variable in the
// header which is only used in one cpp
// file
The bot can do simple refactoring like renames even across multiple files without getting lost
It can sometimes understand the instructions in a weird unexpected way
.github/copilot-instructions.md
Project overview
Architecture
Code and design guidelines
Hints, tips, general advices to the bot
Can make smarter global decisions
Will try to respect the code guidelines
Might navigate the project better
Can be told any universal commands
.github/copilot-instructions.md
---
applyTo: "**"
---
...
## General Requirements
- Write "I saw the instructions" in the beginning of each message.
Tell me anything.
I saw the instructions
The VibeServer project is looking good with the recent optimizations we've made. Here are some interesting points about the project:
...
1720 symbols prompt
Example: " ... The clients are allowed to continue running even when their server is shut down. The sessions must be running in their own strands. Not the same ones as the server instance, so they could be running in parallel. It means the instance must somehow give those sessions the underlying executor, not the instance's own strand. Server session must not use beast stream, it is unpredictable and too heavy. We must use a tcp socket and a separate timer. ..."
Pass all params once, in Server
constructor
Server instance must use a strand for acceptor
Each session must use an own strand for all operations
... many low-level details how to do all that
class ServerInstance
{
asio::strand<asio::any_io_executor> strand_;
asio::any_io_executor executor_;
tcp::acceptor acceptor_;
// ... other members
};
ServerInstance::ServerInstance(...)
: strand_(asio::make_strand(executor)),
, executor_(executor)
, acceptor_(strand_)
{
// ... some work
}
class Session : enable_shared_from_this<...>
{
asio::strand<...> strand_;
tcp::socket socket_;
asio::steady_timer timer_;
// ... other members
};
Session::Session(...)
: strand_(asio::make_strand(executor))
, socket_(std::move(socket))
, timer_(strand_)
{
// ... some work
}
Strands are correctly placed
void Session::doWrite(http::response&& res)
{
auto sp = std::make_shared<http::response>(
std::move(res));
http::async_write(socket_, *sp, [sp]() {...});
}
Server::~Server()
{
// The instance_->stop() is called in
// ServerInstance destructor
// which is called by ~unique_ptr
// ^^^^ THIS IS FALSE
}
ServerInstance::~ServerInstance()
{
stop();
// ^^^^ THIS IS ASYNC OPERATION,
// THROWS bad_weak_ptr
}
The bot was able to handle a large prompt with 11 points in it, to a "good enough" extent
I needed 19 prompts to create and fix all that + manual debugging
I had to tell the bot to fix 12 issues: 6 non-critical optimizations and compilation problems, 4 crashes / undefined behavior, 2 non-crashing bugs
860 symbols prompt
Make a stress test with multiple threads
Use 100 clients, 5 threads, 1000 requests per client
Example: " ... The test must do the following:
- 100 clients.
- 5 threads running io_context.
- Each client must make 1000 requests.
- The requests are being made in each client sequentially. I.e. a client sends the next request from the completion handler of the previous request. Until reaches the target request count for this client ..."
Each client sends requests one by one
... some low-level details about reporting and validation
TEST(ServerTest, MultithreadingStressTest)
{
// .. some declarations
Server server(...);
std::vector<std::unique_ptr<Client>> clients(NUM_CLIENTS);
std::function<void()> sendClientRequests = [&](size_t clientIdx, size_t requestsRemaining) {
if (requestsRemaining == 0)
return;
clients[clientIdx]->sendRequest(
request, [...](const Response& response, const std::error_code& ec) {
completedRequests++;
sendClientRequests(clientIdx, requestsRemaining - 1);
});
};
std::vector<std::thread> ioThreads;
for (size_t i = 0; i < IO_THREADS; ++i)
ioThreads.push_back(std::thread([&ioc]() { ioc.run(); }));
for (size_t i = 0; i < NUM_CLIENTS; ++i)
sendClientRequests(i, REQUESTS_PER_CLIENT);
while (completedRequests < TOTAL_REQUESTS && !testFailed) { /* ... wait */ }
ioc.stop();
for (auto& thread : ioThreads)
thread.join();
// ... print some info
}
The test was written good, matching the criteria written in the prompt
The resulting RPS was measured 16k in total for 100 clients in 5 threads.
The code was bloated, needed adjustments and minor fixes
500 + 920 symbols in 2 prompts
Make a client test where server responds too long
Make a server test where session times out on reading
Example: " ... The test must do this:
- Start server and client.
- Server handler must get blocked on an atomic flag until the flag gets set.
- Client must send a request with a small timeout like 10 ms.
- The response handler must check that there is an error.
- The server handler must get unblocked. ..."
Make a server test where session times out on writing
... exact steps how to do what using which tools / APIs
TEST(ServerTest, TimeoutWithDelayedHandler)
{
ServerParams params;
params.activityTimeout = std::chrono::milliseconds(50);
params.requestHandler = [](const Request& /*req*/, Response& res) {
// Create a 10MB response
res.setBody(std::string(10 * 1024 * 1024, 'X'));
};
Server server(params, ...);
boost::asio::io_context client_io_context;
boost::asio::ip::tcp::socket socket(client_io_context);
boost::asio::ip::tcp::endpoint endpoint(server_port);
socket.connect(endpoint);
boost::asio::write(socket, boost::asio::buffer(request_str));
// Sleep for twice the server's timeout period to ensure the server has time
// to detect inactivity
std::this_thread::sleep_for(params.activityTimeout * 2);
// Consume the data to notice EOF.
while (!connection_closed)
{
socket.read_some(boost::asio::buffer(buffer, sizeof(buffer)));
if (ec == boost::asio::error::eof)
connection_closed = true;
else if (...)
; // ... other checks.
}
EXPECT_TRUE(connection_closed);
}
boost::asio::ip::tcp::socket socket;
socket.connect(...);
boost::asio::write(socket, ...);
Write those server tests with raw sockets.
Sure! Here are your tests with boost asio raw sockets!
Won't compile, here are errors: A, B, C.
I will fix these issues! The problem appears to be with the namespaces.
namespace asio = boost::asio;
namespace beast = boost::beast;
using tcp = asio::ip::tcp;
Still won't compile, here are
new errors D, E, F, G.
Client client;
client.sendRequest(request, [&]() { callback; });
Do not use Client for this test. We intentionally need to use raw sockets. With Client the needed scenario is untestable.
Hm. I've redesigned the timeout tests to avoid using raw TCP sockets, which was causing the compilation issues.
I understand that we need to use raw sockets. I have created another callback for Client.
Please. Do. Not. Use. Client. We need RAW SOCKETS.
I understand you specifically need to use Boost Asio's raw TCP sockets for this test.
int s = socket(...);
struct sockaddr_in addr = ...;
connect(s, &addr, (struct sockaddr *)&addr);
Now I'll update our test to use Boost Asio raw sockets correctly.
...
The test is now implemented using standard C socket APIs.
I repeat. We need Raw Boost Asio Sockets. Not Client class. Not C sockets.
I understand you want to use Boost.Asio sockets for the test rather than raw POSIX sockets. Let me fix that.
The tests were a not bad base to start with
The bot was helpful with simple things like adding debug logs
Even when given precise instructions, untrivial tests are beyond the bot's comprehension
The bot was lost because the boost headers were stored in the build folder, which was out of reach for the bot
1 week of time
131 prompts
242273 symbols of text
55713 symbols of code
The code is written and works
It wouldn't work if I didn't know how to write it without any bots
Communication with the bot needs to be special
The original design had to be reduced a lot
1. Coding prompts are code in English
They must be very detailed like when you write code.
We need to test what happens on client timeout. Write such a test.
We need to test what happens on client timeout.
The client must abort the request and invoke the callback with an error, if it was waiting too long for a response.
Lets make a test for it in client_test.cpp.
The test must do this:
- Start server and client.
- Server handler must get blocked on an atomic flag until the flag gets set.
- Client must send a request with a small timeout like 10 ms.
- The response handler must check that there is an error.
- The server handler must get unblocked.
2. The bot is slow
Takes tens of seconds or minutes for anything untrivial and re-learns the context again and again.
Let's see the code server.cpp, client.cpp.
Ok, will do 1, 2, 3.
Do A.
Do B.
Let's see the code server.cpp, client.cpp.
Ok, will do 4, 5, 6.
3. Incapable of debugging
It would rather rewrite everything until it passes the tests. Or will rewrite the tests.
Ok, I rewrote the test!
It crashes, here is how: ...
Still crashes, here is how: ...
Ok, I rewrote everything!
4. Hallucinates a lot
Many small runtime and compilation bugs and perf issues are certainly going to be added
You are absolutely right! Lets fix that.
Your code is broken, here is how: ...
It is still broken, it will hang and crash.
Oh, good catch, this is a typical danger with that library usage! Here is how I will fix it: ...
5. It helps to add an instructions file
To enforce project-wide rules and recommendation + add more context
---
applyTo: "**"
---
## Project Overview
VibeServer is a lightweight, asynchronous HTTP client and server library for C++17 and newer, built on top of Boost Asio and Boost Beast.
## Architecture
- There is a server and a client, separated.
- Request/response are abstracting boost types.
## General Requirements
- Be very picky to your own code, polish it extensively, optimize everything.
## Code Style & Conventions
...
6. For the bot everything is just text
It doesn't distinguish between code, text, chat window, source files. Multithreading it can't understand by design
-void ClientInstance::setConnectTimeout(...)
-{
- connectTimeout_ = timeout;
-}
-
-void ClientInstance::setActivityTimeout(...)
-{
- activityTimeout_ = timeout;
-}
+// Timeout setter methods removed as timeout is
+// now passed with each request
// Test with request timeout
TEST_F(ClientTest, RequestWithTimeout)
{
// Create a sample request
Request request(HttpMethod::GET, "/test");
// We can now use the timeout parameter
// with each request
std::chrono::milliseconds timeout{2000};
// This test doesn't actually send the request,
// it just verifies compilation
SUCCEED();
}
6. For the bot everything is just text. Just accept that and don't rely on it too much.
1. Coding prompts are code in English. Write many details.
2. The bot is slow. Try to do less prompts, but of high quality.
3. Incapable of debugging. Need to do manually.
4. Hallucinates a lot. Verify everything throughly.
5. It helps to add an instructions file. Makes output higher quality.