Vibecoding an HTTP
client-server library

Vladislav Shpilevoy

Virtual Minds

* visibly vibing

Plan

Goal

Task

Setup

Development

Summary

Goal

Goal

Find out how good is LLM at coding

Discover LLM limitations in coding

Learn how to use LLM efficiently

Task

Task

boost::asio + boost::beast

Implement an HTTP client-server library

All operations asynchronous

Multithreading

Maximum focus on performance

Maximum test coverage

No dependencies besides boost

Only LLM is allowed to write code. I must not intervene.

Only HTTP 1.x

No SSL

Setup

Setup

Claude Sonnet 3.7

VSCode / GitHub Copilot

New chat

Task prompt

Review output

Make it compile

Make it pass tests

Close chat

Launch

Launch

2800 symbols initial prompt

Example: "... Any sort of third-party dependencies like boost must be installed and visible only for this project. Not globally in the system. It is up to you how to achieve that. Conda or some sort of venv for isolation of the entire environment, or conan, or alike. You can only globally rely on the compiler, the standard library, cmake, and make ..."

HTTP client-server library

C++17

Minimalistic public API

... and other points

vibeserver/                 

- cmake/             

- examples/          

- include/vibeserver/

- src/vibeserver/    

- tests/             

Launch

Code samples

Launch

Code samples - client API

class ClientImpl;

class Client {
public:
    using RequestCallback = std::function<void(const Response&, const std::error_code&)>;
    using Ptr = std::unique_ptr<Client>;

    static Ptr create(boost::asio::any_io_executor& executor, const std::string& host, const std::string& port);
    
    virtual ~Client() = default;

    // Asynchronously send a request and receive a response
    virtual void sendRequest(const Request& request, RequestCallback callback) = 0;
    
    // Set connection timeout
    virtual void setConnectTimeout(std::chrono::milliseconds timeout) = 0;
    
    // Set activity timeout
    virtual void setActivityTimeout(std::chrono::milliseconds timeout) = 0;
};

All methods in the header are virtual

Launch

Code samples - server implementation

class ServerImpl : public Server, public std::enable_shared_from_this<ServerImpl> {
    void doAccept() {
        acceptor_.async_accept(
            asio::make_strand(executor_),
            beast::bind_front_handler(
                &ServerImpl::onAccept,
                shared_from_this()));
    }

    void onAccept(beast::error_code ec, tcp::socket socket) {
        if (ec) {
            logging::Logger::debug("Accept failed: " + ec.message());
        } else {
            std::make_shared<Session>(
                std::move(socket),
                requestHandler_,
                activityTimeout_)
                ->start();
        }
        if (acceptor_.is_open()) {
            doAccept();
        }
    }

    asio::any_io_executor& executor_;
    tcp::acceptor acceptor_;

};

Async accept

New session for each client

Each session is running in an own thread-safe strand

Launch

Highlights

Launch

Highlights: CMake for dependencies

Summary: "All third-party deps must be local and managed by conan / conda / venv / other-tool up to your choice"

Launch

Highlights: CMake for dependencies

The bot has decided to use CMake as a "package manager"

FetchContent_Declare(
  Boost
  URL ${BOOST_URL}
  URL_HASH SHA256=954a01219b....
)
set(FETCHCONTENT_QUIET OFF)
FetchContent_MakeAvailable(Boost)

FetchContent_GetProperties(Boost SOURCE_DIR BOOST_SOURCE_DIR)

set(Boost_INCLUDE_DIRS ${BOOST_SOURCE_DIR})

It used an old boost version (1.82 instead of the latest 1.89)

It made the package checksum up

Entered a loop of wrong usage of add_library() and making boost CMake variables up

Launch

Highlights: code quality

Summary: "The code must be paranoid about performance and be minimalistic but clear, without narrative comments"

Launch

Highlights: code quality

http::async_write(connection->stream, req,
    [this, connection, callback](...) {
        if (ec) {
            callback({}, ec);
            return;
        }
        onWrite(connection, callback);
    });
// Resolve the host name and port
resolver_.async_resolve(...)

// Set the timeout for the operation
beast::get_lowest_layer(connection->stream).expires_after(...);

// Copy body
res.body() = response.body();

// Keep-alive handling
res.keep_alive(req_.keep_alive());
bool Response::hasHeader(const std::string& name) const
{
    return headers_.find(name) != headers_.end();
}

Async boost IO is correctly used, mostly

Public API is hiding internals good

Many useless narrative comments

Completely missed string_view existence

p1 - Interfaces

p1 - Interfaces

980 symbols prompt

Get rid of virtual functions

Unified and short namespace 'vibe'

Example: "... The implementation must be encapsulated in the cpp files using this pattern: the public class is in the header, its only member is a unique_ptr storing an "instance" class like Server and ServerInstance. The instance type is forward-declared in the header. But defined in the cpp file. It is created in the public class object constructor. The public class has non-virtual methods which just forward all their arguments in the cpp file to the instance class ..."

p1 - Interfaces

class ClientInstance;

class Client
{
 public:
	using RequestCallback = std::function<void(const Response&, const std::error_code&)>;

	Client(boost::asio::any_io_executor executor, const std::string& host, const std::string& port);
	~Client();

	void sendRequest(const Request& request, RequestCallback callback);
	void setConnectTimeout(std::chrono::milliseconds timeout);
	void setActivityTimeout(std::chrono::milliseconds timeout);

 private:
	std::unique_ptr<ClientInstance> instance_;
};

Each session is running in an own thread-safe strand

p1 - Interfaces

Testing

p1 - Interfaces

Testing

The examples didn't run

The existing tests didn't cover anything

class ClientTest : public ::testing::Test
{
 protected:
	void SetUp() override
	{
		// Create client
		client_ = std::make_unique<Client>(ioc_.get_executor(), "example.com", "80");
	}

	boost::asio::io_context ioc_;
	std::unique_ptr<vibe::Client> client_;
};

// Basic client creation test
TEST_F(ClientTest, CreateAndDestroy)
{
	// Client is already created in SetUp
	// It will be destroyed in TearDown
	SUCCEED();
}

p1 - Interfaces

Testing

Add new tests to cover client-server communication

Example: "... Your tests don't have any server + client cases. Write a basic test which creates a server, a client, connects the client to the server, and ensures a request/response works. ..."

p1 - Interfaces

Testing

// Test server and client communication
TEST(ServerTest, ServerClientCommunication)
{
	boost::asio::io_context ioc;
	Server server(ioc.get_executor(), "127.0.0.1", 0);

	// ... preparations

	Client client(ioc.get_executor(), "127.0.0.1", std::to_string(serverPort));
	Request request(HttpMethod::POST, "/test/endpoint");
	request.setBody("Hello, Server!");
	client.sendRequest(request, [&](const Response& response, std::error_code ec) {
		// ... handling
	});

	ioc.run();

	// ... verification
	EXPECT_TRUE(...);
	EXPECT_EQ(...);
}

p1 - Interfaces

Testing

I am getting a crash, here is <output...>

The test was crashing. The bot would try to rewrite everything instead of debugging.

I will help you! I've fixed several issues in the test that could lead to a crash.

Still crashing. Help me to debug.

Lets dig deeper. I have completely rewritten the test. Here is 7 improvements and 4 crash reasons.

Don't rewrite everything, we don't know what the issue is. Lets add debug prints.

You're absolutely right. Let's take a more methodical approach by adding debug output statements.

Please, stop doing anything except adding prints. Only add them and leave the rest.

p1 - Interfaces

Testing

I have debugged it myself, told the problem to the bot, and it was fixed.

void ClientInstance::onConnect(...)
{
	http::request<http::string_body> req{...};
	http::async_write(req, ...);
}

Here is the bug - object 'req' is deleted while used in async operation.

You've identified the exact issue! This is a classic problem with asynchronous operations - the stack-allocated `req` object in `onConnect` is being destroyed when the function returns, but the async operation still needs it.

... <1600 symbols of explanation>

p1 - Interfaces

Highlights

p1 - Interfaces

Highlights

TEST(ServerTest, ServerClientCommunication)
{
	// ...
}

TEST(ServerTest, ServerCustomResponse)
{
	// ...
}
void ClientInstance::onConnect(...)
{
	http::request<http::string_body> req{...};
	http::async_write(req, ...);
    // ^^^ 'req' is deleted later, while stil in use
}
http::request<http::string_body> req{
	static_cast<http::verb>(request.method())
}; //           ^^^^^^^ those enums are incompatible

Helps to prepare the ground by generating lots of raw code for starting fast, then polish

Can write untrivial tests if the prompt is very detailed

Has Tourette syndrome - can't shut up and do only what it is told

Completely incapable of debugging

p2 - Single client timeout

p2 - Single client timeout

390 symbols prompt

Client creates connection for each request

Need to make the timeout also per request

Full prompt: "We need to rework the Client. Currently it has separate connection timeout and activity timeout. But it doesn't make much sense now. The idea was that the connection would be one and it would stay active. If it doesn't then can as well merge these timeouts into one, and pass that timeout with each request. Then you should be able to delete setConnectTimeout and setActivityTimeout methods."

p2 - Single client timeout

-       void sendRequest(const Request& request, RequestCallback callback);
-       void setConnectTimeout(std::chrono::milliseconds timeout);
-       void setActivityTimeout(std::chrono::milliseconds timeout);
+       void sendRequest(const Request& request,
+                        RequestCallback callback,
+                        std::chrono::milliseconds timeout = std::chrono::seconds(30));
void ClientInstance::onResolve(...)
{
	connection->stream.expires_after(timeout);
	connect(...);
}

void ClientInstance::onConnect(...)
{
	connection->stream.expires_after(timeout);
	write(...);
}

void ClientInstance::onWrite(...)
{
	connection->stream.expires_after(timeout);
	read(...);
}

Timeout is reset after each action

p2 - Single client timeout

Highlights

p2 - Single client timeout

Highlights

// Test with request timeout
TEST_F(ClientTest, RequestWithTimeout)
{
	// Create a sample request
	Request request(HttpMethod::GET, "/test");

	// We can now use the timeout parameter
    // with each request
	std::chrono::milliseconds timeout{2000};

	// This test doesn't actually send the request,
    // it just verifies compilation
	SUCCEED();
}

The task is solved - timeout is now per-request

Didn't need much hand holding - simple tasks like this it can do relatively well

Don't trust it to write tests on its own

p3 - Client strands

p3 - Client strands

840 symbols prompt

Timeout needs to become per request, not per operation

Introduce a per request timer object

Example: "... Timeout now is per-operation. Need to make it per--connection. Give Connection its own resolver, timer, socket, delete the tcp_stream, launch the timer on Connection start. Cancel everything when the timer expires. But cancel the timer, when the request succeeds. Use shared_from_this() to secure the lifetime of the Connection  ..."

Serialize timer and IO events using a boost strand

p3 - Client strands

struct Connection : enable_shared_from_this<...>
{
	asio::strand<...> strand_;
	tcp::resolver resolver_;
	tcp::socket socket_;
	asio::steady_timer timer_;
	// ... other members
};

Connection::Connection(asio::any_io_executor executor)
	: strand_(executor)
	, resolver_(strand_)
	, socket_  (strand_)
	, timer_   (strand_)
{
	timer_.expires_after(timeout);
	// ... other work
}

void Connection::onTimeout()
{
	if (cancelled_)
	{
		return;
	}
	doCancel();
	callback_({}, asio::error::operation_aborted);
}

All work is running on the strand

Timeout is correctly aborting everything

p3 - Client strands

Highlights

p3 - Client strands

Highlights

The task is solved - timeout is per-request and request has a strand

Done well, only minor polishing and bug fixes were done afterwards

The bot is the most efficient when works in a single file on an isolated task

It makes many small bugs. Very rare when an untrivial patch would have none

p4 - Code cleanup

p4 - Code cleanup

380 symbols prompt

Remove all namespaces except for 'vibe'

Remove 'Logger' class, turn it into free functions

Full prompt: "We need to clean up the styling a little bit. You need to do the following:
 

- Remove the 'logging' namespace. Only 'vibe' namespace must remain. The Logger class must go away. Only free functions must remain and must be named like logDebug, logInfo.

- Make all non-third-party headers includes via "", not <>. And separate them from the system inclusions with an empty line."

Use include <...> for third-party headers, and "..." for local ones

p4 - Code cleanup

 #include <string>
 #include <string_view>
 #include <boost/asio/any_io_executor.hpp>
-#include <vibeserver/request.hpp>
-#include <vibeserver/response.hpp>
+
+#include "vibeserver/request.hpp"
+#include "vibeserver/response.hpp"
namespace vibe
{

enum class LogLevel
{
	Debug,
	Info,
	Off
};

void setLogLevel(LogLevel level);
LogLevel getLogLevel();

void logDebug(const std::string& message);
void logInfo(const std::string& message);

}  // namespace vibe

p4 - Code cleanup

Highlights

p4 - Code cleanup

Highlights

	// logging.hpp


	// Private implementation detail
	extern LogLevel logLevel_;
    
    // ^^^ a private global variable in the
    // header which is only used in one cpp
    // file

The bot can do simple refactoring like renames even across multiple files without getting lost

It can sometimes understand the instructions in a weird unexpected way

p5 - Instructions file

p5 - Instructions file

.github/copilot-instructions.md

Project overview

Architecture

Code and design guidelines

Hints, tips, general advices to the bot

Can make smarter global decisions

Will try to respect the code guidelines

Might navigate the project better

Can be told any universal commands

p5 - Instructions file

.github/copilot-instructions.md


---
applyTo: "**"
---

...

## General Requirements

- Write "I saw the instructions" in the beginning of each message.

Tell me anything.

I saw the instructions

The VibeServer project is looking good with the recent optimizations we've made. Here are some interesting points about the project:
...

p6 - Server strands

p6 - Server strands

1720 symbols prompt

Example: " ... The clients are allowed to continue running even when their server is shut down. The sessions must be running in their own strands. Not the same ones as the server instance, so they could be running in parallel. It means the instance must somehow give those sessions the underlying executor, not the instance's own strand. Server session must not use beast stream, it is unpredictable and too heavy. We must use a tcp socket and a separate timer. ..."

Pass all params once, in Server constructor

Server instance must use a strand for acceptor

Each session must use an own strand for all operations

... many low-level details how to do all that

p6 - Server strands

class ServerInstance
{
	asio::strand<asio::any_io_executor> strand_;
	asio::any_io_executor executor_;
	tcp::acceptor acceptor_;
	// ... other members
};

ServerInstance::ServerInstance(...)
	: strand_(asio::make_strand(executor)),
	, executor_(executor)
	, acceptor_(strand_)
{
	// ... some work
}

class Session : enable_shared_from_this<...>
{
	asio::strand<...> strand_;
	tcp::socket socket_;
	asio::steady_timer timer_;
	// ... other members
};

Session::Session(...)
	: strand_(asio::make_strand(executor))
	, socket_(std::move(socket))
	, timer_(strand_)
{
	// ... some work
}

Strands are correctly placed

p6 - Server strands

Highlights

p6 - Server strands

void Session::doWrite(http::response&& res)
{
	auto sp = std::make_shared<http::response>(
    	std::move(res));
	http::async_write(socket_, *sp, [sp]() {...});
}
Server::~Server()
{
	// The instance_->stop() is called in
    // ServerInstance destructor
	// which is called by ~unique_ptr

	// ^^^^ THIS IS FALSE
}
ServerInstance::~ServerInstance()
{
	stop();

	// ^^^^ THIS IS ASYNC OPERATION,
	// THROWS bad_weak_ptr
}

Highlights

The bot was able to handle a large prompt with 11 points in it, to a "good enough" extent

I needed 19 prompts to create and fix all that + manual debugging

I had to tell the bot to fix 12 issues: 6 non-critical optimizations and compilation problems, 4 crashes / undefined behavior, 2 non-crashing bugs

p7 - Stress test

p7 - Stress test

860 symbols prompt

Make a stress test with multiple threads

Use 100 clients, 5 threads, 1000 requests per client

Example: " ... The test must do the following:
- 100 clients.
- 5 threads running io_context.
- Each client must make 1000 requests.
- The requests are being made in each client sequentially. I.e. a client sends the next request from the completion handler of the previous request. Until reaches the target request count for this client ..."

Each client sends requests one by one

... some low-level details about reporting and validation

p7 - Stress test

TEST(ServerTest, MultithreadingStressTest)
{
	// .. some declarations

	Server server(...);
	std::vector<std::unique_ptr<Client>> clients(NUM_CLIENTS);

	std::function<void()> sendClientRequests = [&](size_t clientIdx, size_t requestsRemaining) {
		if (requestsRemaining == 0)
			return;
		clients[clientIdx]->sendRequest(
		    request, [...](const Response& response, const std::error_code& ec) {
			    completedRequests++;
			    sendClientRequests(clientIdx, requestsRemaining - 1);
		    });
	};

	std::vector<std::thread> ioThreads;
	for (size_t i = 0; i < IO_THREADS; ++i)
		ioThreads.push_back(std::thread([&ioc]() { ioc.run(); }));

	for (size_t i = 0; i < NUM_CLIENTS; ++i)
		sendClientRequests(i, REQUESTS_PER_CLIENT);

	while (completedRequests < TOTAL_REQUESTS && !testFailed) { /* ... wait */ }

	ioc.stop();
	for (auto& thread : ioThreads)
			thread.join();

	// ... print some info
}

p7 - Stress test

Highlights

p7 - Stress test

Highlights

The test was written good, matching the criteria written in the prompt

The resulting RPS was measured 16k in total for 100 clients in 5 threads.

The code was bloated, needed adjustments and minor fixes

p8 - Timeout tests

p8 - Timeout tests

500 + 920 symbols in 2 prompts

Make a client test where server responds too long

Make a server test where session times out on reading

Example: " ... The test must do this:
- Start server and client.
- Server handler must get blocked on an atomic flag until the flag gets set.
- Client must send a request with a small timeout like 10 ms.
- The response handler must check that there is an error.
- The server handler must get unblocked. ..."

Make a server test where session times out on writing

... exact steps how to do what using which tools / APIs

p8 - Timeout tests

TEST(ServerTest, TimeoutWithDelayedHandler)
{
	ServerParams params;
	params.activityTimeout = std::chrono::milliseconds(50);
	params.requestHandler = [](const Request& /*req*/, Response& res) {
		// Create a 10MB response
		res.setBody(std::string(10 * 1024 * 1024, 'X'));
	};

	Server server(params, ...);

	boost::asio::io_context client_io_context;
	boost::asio::ip::tcp::socket socket(client_io_context);
	boost::asio::ip::tcp::endpoint endpoint(server_port);
	socket.connect(endpoint);
	boost::asio::write(socket, boost::asio::buffer(request_str));

	// Sleep for twice the server's timeout period to ensure the server has time
	// to detect inactivity
	std::this_thread::sleep_for(params.activityTimeout * 2);

	// Consume the data to notice EOF.
	while (!connection_closed)
	{
		socket.read_some(boost::asio::buffer(buffer, sizeof(buffer)));
		if (ec == boost::asio::error::eof)
			connection_closed = true;
		else if (...)
			; // ... other checks.
	}
	EXPECT_TRUE(connection_closed);
}

p8 - Timeout tests

boost::asio::ip::tcp::socket socket;
socket.connect(...);
boost::asio::write(socket, ...);

Write those server tests with raw sockets.

Sure! Here are your tests with boost asio raw sockets!

Won't compile, here are errors: A, B, C.

I will fix these issues! The problem appears to be with the namespaces.

namespace asio = boost::asio;

namespace beast = boost::beast;

using tcp = asio::ip::tcp;

Still won't compile, here are
new errors D, E, F, G.

p8 - Timeout tests

Client client;

client.sendRequest(request, [&]() { callback; });

Do not use Client for this test. We intentionally need to use raw sockets. With Client the needed scenario is untestable.

Hm. I've redesigned the timeout tests to avoid using raw TCP sockets, which was causing the compilation issues.

I understand that we need to use raw sockets. I have created another callback for Client.

Please. Do. Not. Use. Client. We need RAW SOCKETS.

I understand you specifically need to use Boost Asio's raw TCP sockets for this test.

p8 - Timeout tests

int s = socket(...);

struct sockaddr_in addr = ...;

connect(s, &addr, (struct sockaddr *)&addr);

Now I'll update our test to use Boost Asio raw sockets correctly.

...
The test is now implemented using standard C socket APIs.

I repeat. We need Raw Boost Asio Sockets. Not Client class. Not C sockets.

 I understand you want to use Boost.Asio sockets for the test rather than raw POSIX sockets. Let me fix that.

p8 - Timeout tests

Highlights

p8 - Timeout tests

Highlights

The tests were a not bad base to start with

The bot was helpful with simple things like adding debug logs

Even when given precise instructions, untrivial tests are beyond the bot's comprehension

The bot was lost because the boost headers were stored in the build folder, which was out of reach for the bot

Summary

Summary

1 week of time

131 prompts

242273 symbols of text

55713 symbols of code

The code is written and works

It wouldn't work if I didn't know how to write it without any bots

Communication with the bot needs to be special

The original design had to be reduced a lot

Learnings

Learnings

1. Coding prompts are code in English

They must be very detailed like when you write code.

We need to test what happens on client timeout. Write such a test.

We need to test what happens on client timeout.

The client must abort the request and invoke the callback with an error, if it was waiting too long for a response.

Lets make a test for it in client_test.cpp.

The test must do this:
- Start server and client.
- Server handler must get blocked on an atomic flag until the flag gets set.
- Client must send a request with a small timeout like 10 ms.
- The response handler must check that there is an error.
- The server handler must get unblocked.

Learnings

2. The bot is slow

Takes tens of seconds or minutes for anything untrivial and re-learns the context again and again.

Let's see the code server.cpp, client.cpp.


Ok, will do 1, 2, 3.

Do A.

Do B.

Let's see the code server.cpp, client.cpp.

Ok, will do 4, 5, 6.

Learnings

3. Incapable of debugging

It would rather rewrite everything until it passes the tests. Or will rewrite the tests.

Ok, I rewrote the test!

It crashes, here is how: ...

Still crashes, here is how: ...

Ok, I rewrote everything!

Learnings

4. Hallucinates a lot

Many small runtime and compilation bugs and perf issues are certainly going to be added

You are absolutely right! Lets fix that.

Your code is broken, here is how: ...

It is still broken, it will hang and crash.

Oh, good catch, this is a typical danger with that library usage! Here is how I will fix it: ...

Learnings

5. It helps to add an instructions file

To enforce project-wide rules and recommendation + add more context

---

applyTo: "**"
---

## Project Overview
VibeServer is a lightweight, asynchronous HTTP client and server library for C++17 and newer, built on top of Boost Asio and Boost Beast.

## Architecture
- There is a server and a client, separated.
- Request/response are abstracting boost types.
 

## General Requirements

- Be very picky to your own code, polish it extensively, optimize everything.

## Code Style & Conventions
...

Learnings

6. For the bot everything is just text

It doesn't distinguish between code, text, chat window, source files. Multithreading it can't understand by design

-void ClientInstance::setConnectTimeout(...)
-{
-       connectTimeout_ = timeout;
-}
-
-void ClientInstance::setActivityTimeout(...)
-{
-       activityTimeout_ = timeout;
-}
+// Timeout setter methods removed as timeout is
+// now passed with each request
// Test with request timeout
TEST_F(ClientTest, RequestWithTimeout)
{
	// Create a sample request
	Request request(HttpMethod::GET, "/test");

	// We can now use the timeout parameter
    // with each request
	std::chrono::milliseconds timeout{2000};

	// This test doesn't actually send the request,
    // it just verifies compilation
	SUCCEED();
}

Learnings

6. For the bot everything is just text. Just accept that and don't rely on it too much.

1. Coding prompts are code in English. Write many details.

2. The bot is slow. Try to do less prompts, but of high quality.

3. Incapable of debugging. Need to do manually.

4. Hallucinates a lot. Verify everything throughly.

5. It helps to add an instructions file. Makes output higher quality.