Evolving Software: Embracing AI-Driven Development

2023-04-08

Introduction

In this post, we'll explore a system of software development that allows for evolution and greater responsibilities as AI capabilities and context grow. Unlike emergent AI functions, where native functions interact with AI capabilities through an interface, this approach enables AI to build and maintain its own responsibilities based on a set of goals or directives.

Initial thoughts

The AI generative model/system would have a set of goals and requirements it would fulfill, it would build the initial version of the code, or spawn sub-AIs to build capabilities for them.

It would handle requirements as they come in, and may even be set up to automatically improve the code by updating to newer libraries, improve performance and create more maintainable code

Starting Small: A Minimal Viable Product

Let's begin with a single function or unit of work, similar to what you'd find in a unit test. The AI generative model would be responsible for this function, but not as a black box. Instead, it would resemble the following:

enum SomeFunctionError {
    ...
}

struct SomeModel {
    ...
}

fn some_function() -> Result<...> {

	let resp = call_api().map_err(|e| SomeFunctionError::ServerErr(e))?;

    let content: SomeModel = resp.content()?;

	return Ok(content)
}

As more requirements arise, the generative system can be informed of them, allowing it to evolve the function or system accordingly:

enum SomeFunctionError {
    ...
}

struct SomeModel {
    ...
}

fn some_function() -> Result<...> {
	let resp = call_api().map_err(|e| SomeFunctionError::ServerErr(e))?;

    let content: SomeModel = resp.content()?;

    log_metrics(content)?; // new

	return Ok(content)
}

The generative model would automatically refresh its context when it cycles, allowing developers to directly modify the code without any runtime magic.

Scaling Up: Introducing More Responsibility

As the capabilities and context of the AI model evolve, abstraction levels can be increased, allowing each AI layer to manage its own capabilities. The hierarchy would look like this:

service has modules which has files.

Each file maintains its own context and responsibility within a module, which itself is a single AI instance. The primary AI module can direct and query sub-AIs for their capabilities, prompting them to fix bugs, add features, and even spawn new AIs for emerging requirements.

Interaction: System Level and Public API

Interaction with the AI should be possible both at the system level and via a public API. Primary engineers can prompt the AI directly, enabling it to update its goals and delegate tasks to its child systems.

Through a public API like GitHub, the AI would have its own user account, allowing developers to mention or assign it to issues. The AI would then handle the issue directly, offering help, closing it, or fixing the submitted bug.

A Thought Experiment: Real-World Viability

While this concept requires testing in real-world scenarios, tools like AutoGPT and GPT4All could potentially be adapted for this purpose. The groundwork laid by AutoGPT makes integration with existing systems like Git, GitHub, and web search feasible, along with delegation and supervision tasks.

The Future of AI-Managed Software Systems

An automated AI-managed software system may soon become a reality, and this post outlines a potential model for incrementally increasing AI responsibility as its capabilities grow. Although AI models are currently intelligent and capable, their context and long-term memory are not as mature, making a gradual model more suitable for implementation.

A practical example will follow as I experiment more.

Reflecting on the AI Experience

Working with these AI models has yielded surprising results. Initially, i anticipated that AI would generate obscure and difficult-to-maintain code. However, the opposite has proven true: AI can create incredibly readable and maintainable code. The key is providing concise and directed requirements, as the AI is quite adept at discerning nuances within them and taking appropriate action.

The primary challenges i face involve context limitations and context expansion (acquiring new knowledge). The current context for models like ChatGPT or GPT-4 is quite restricted, with a maximum of 32k tokens (around 20k words). This constraint must accommodate all the directives driving the generative software system, its acquired knowledge, and any new requirements.

The central issue is the lack of an easy way for AI to gain knowledge without exceeding its context cache. While GPT could read an entire library's source code to understand it, doing so would result in a biased perspective based on that specific implementation. Alternatively, GPT could read a library's API, but there is no standard method that's general enough for our use case. Developing an ingestion function for each language, package manager, and documentation system would be necessary.

A practical solution involves using AI to optimize context for another AI. In other words, one AI fetches and digests the documentation, then compresses it as succinctly as possible for another AI to use. While this approach may not be perfect, as the AI is not specifically designed to optimize for another AI, it offers a promising workaround.

Long-term storage is another viable option that i plan to explore. However, its effectiveness in practice and the extent of context it can restore remain to be seen.