Emerging Complex Programs 

The complexity science of software evolution and emergence

Complex System

"a system in which large networks of components with no central control and simple rules of operation give rise to complex collective behavior, sophisticated information processing, and adaptation via learning or evolution" ~Mitchell, M.

Complex System (alternative def.)

"a system that exhibits nontrivial emergent and self-organizing behaviors"

Properties common to Complex Systems

• Simple Components (or agents)
• Nonlinearity and Complex Interactions
• Decentralization
• Emergency
  • Hierarchical Organization
  • Information Processing
  • Dynamics
  • Evolution and Learning

Software Engineering & Deep Learning

Background

Neural Network

Are Neural Networks Turing Complete?

Algorithm

Program

ON THE TURING COMPLETENESS OF MODERN NEURAL NETWORK ARCHITECTURES (ICLR'19 Perez, et al.)

For any arbitrary function F(x), there exists a Neural Network that closely approximates it for any input x​

For any arbitrary Program P(x), there exists a Neural Network that closely approximates it for any input x​

Source Code Generative Agents

Deep Learning agents that synthesize source code. Those agents might perform other SE tasks so that they are unsupervised multi-taskers.

 Source Code Generative Agent

 Source Code

 Source Code

 Fine-Tunned Agent

 Source Code

SE Task

Supervised SE Task

What if the Generative Agents interact among each other in a controlled environment?

Complexity Science applied on Source Code Generative Agents

Enhanced supervised task?

Synthesized Code

Generative Agent

Fine-Tuned Agent

Output

Input

Enhanced synthesized source code?

At a Larger Scale the behavior emerge (Power Law)

Can the generated source code produce a generative agent?

Self-Replication (?)

What if the self-replicated agent (by emergency) generates better behavior (i.e. classification accuracy)?

Enhanced task?

Instead of transmitting typical outputs (source code), now the agents also share learned configurations (meta-data)

Emerging Complex Programs is feasible under tensor manipulations and simple local rules 

Deep Software Engineering via Complexity Science Analysis and Design

Towards Self-Maintenance Programs

• Property 1: Simple Components or agents (simple relative to the whole system)

Auto-regressive, adversarial, or autoenconder architectures

Trained Generative Agent

Trained Fine-Tuned Agent

Deep Neural Classifier

• Property 2: Nonlinearity and Complex Interactions (synergy)

Better Performance

Generative Agents are sensitive to initial conditions (hyper-parameters) and inputs

Fine-Tune Strategy 1

Fine-Tune Strategy 2

Worst Performance

• Property 3: Descentralization or no central control 

• Property 4: Emergency

• Property 4: Emergency

• Property 4: Emergency

We can make source code Generative Agents more sophisticated. If each individual unit includes an independent reasoning unit, then agents can interact with each other by taking individual decisions (e.g., game theory, Q-Function, and reinforcement)

Reasoning Units could control for maintenance tasks of each Neural Net. In other words, these nets are ablet to self-maintain the entire architecture (adaptability strategies)

Recall the three types of reasoning process?

Each Generative Agent can be considered "intelligent" when a reasoning is included...

Each Generative Agent can be considered "intelligent" when a reasoning is included...

• Property 4: Emergency with "Intelligent Agents"