Loading...
Loading...
Loading...
The provided code implements a flexible framework for creating Gymnasium environments powered by state machines and integrates a language model (like GPT-4) to enable an agent to perform reasoning and actions in a ReAct (Reasoning and Acting) paradigm. Below is a detailed explanation of each component in the code:
The provided code implements a flexible framework for creating Gymnasium environments powered by state machines and integrates a language model (like GPT-4) to enable an agent to perform reasoning and actions in a ReAct (Reasoning and Acting) paradigm. Below is a detailed explanation of each component in the code:
---
## **1. StateMachineEnv Class**
**Purpose**: This class extends `gym.Env` and serves as a base for creating Gymnasium environments that use state machines. It leverages the `transitions` library to manage state transitions and supports custom callbacks for transitions.
### **Key Components:**
- **Initialization (`__init__` method)**:
- **Parameters**:
- `states`: List of possible states in the environment.
- `actions`: Dictionary mapping action indices to triggers or action definitions.
- `initial_state`: The starting state of the environment.
- `transitions`: List of transition dictionaries defining how states change.
- `observation_space`: The observation space as defined by Gymnasium.
- `callbacks`: Optional dictionary of callbacks for transitions.
- `render_mode`: Mode for rendering the environment.
- `max_steps`: Maximum number of steps before truncation.
- **Functionality**:
- Initializes the environment's action space and observation space.
- Sets up the state machine using the `transitions` library.
- Adds transitions with associated callbacks using the `_add_transitions` method.
- Initializes the observation and sets up rendering.
- **Adding Transitions (`_add_transitions` method)**:
- Iterates over the provided transitions and incorporates any specified callbacks.
- Supports conditions, before, and after callbacks for each transition.
- **Observation Handling**:
- `_update_observation`: Placeholder method to update the observation based on the current state.
- `get_observation`: Returns the current observation.
- **Reward and Termination**:
- `compute_reward`: Computes the reward for the current state and action.
- `is_terminated`: Checks if the episode should terminate.
- **Reset and Step Methods**:
- `reset`: Resets the environment to the initial state and observation.
- `step`: Processes an action, performs state transitions, updates observations, computes rewards, and checks for termination.
### **Example Usage in the Code**:
The `StateMachineEnv` class is used as a foundational environment that other classes can build upon, providing a structured way to manage complex environments with multiple states and transitions.
---
## **2. FunctionalStateEnvironment Class**
**Purpose**: Provides a functional approach to state management by using immutable contexts and state handlers. It wraps around `StateMachineEnv` and allows for pure function computations of the next state.
### **Key Components:**
- **StateContext Data Class**:
- An immutable data class that holds the state data, including the current state, data dictionary, and observation.
- **Initialization**:
- Accepts parameters similar to `StateMachineEnv` but also includes `state_handlers`, a dictionary mapping state names to handler functions.
- Initializes an instance of `StateMachineEnv`.
- Sets up the initial context using `StateContext`.
- **Applying Actions (`apply_action` method)**:
- Executes a step in the `StateMachineEnv`.
- Retrieves the appropriate state handler based on the current state.
- Applies the state handler to compute the next context.
- Updates the internal context with the new state.
### **Example Usage in the Code**:
The `FunctionalStateEnvironment` class serves as a bridge between the state machine environment and functional programming practices, enabling easier reasoning about state changes and side effects.
---
## **3. ReactEnvironment Class**
**Purpose**: Implements the ReAct paradigm by integrating an OpenAI language model (e.g., GPT-4) into the environment. It allows an agent to "think," "act," and "observe" in a loop until a task is completed.
### **Key Components:**
- **ReactState and ReactContext Data Classes**:
- `ReactState`: Holds the agent's thought, action, observation, and any additional context.
- `ReactContext`: Extends `StateContext` to include a history of `ReactState` instances and the current `ReactState`.
- **Initialization**:
- Sets up states (`THINKING`, `ACTING`, `OBSERVING`, `FINISHED`), actions, and transitions specific to the ReAct paradigm.
- Accepts `task_description`, `available_actions`, and an `observation_formatter` function.
- Initializes the base `FunctionalStateEnvironment` with appropriate parameters.
- **State Handlers**:
- **_handle_thinking**:
- Creates a prompt based on the task description, available actions, and the agent's history.
- Uses the OpenAI API to generate the agent's thought.
- Updates the context with the new thought.
- **_handle_acting**:
- Creates a prompt to determine the next action based on the agent's thought.
- Uses the OpenAI API to extract the most appropriate action.
- If the action is invalid, redirects back to thinking.
- Updates the context with the chosen action.
- **_handle_observing**:
- Formats the observation using the `observation_formatter`.
- Updates the context with the new observation.
- **_handle_finished**:
- Handles the completion of the task, potentially performing any cleanup or finalization.
### **Example Usage in the Code**:
The `ReactEnvironment` class embodies the ReAct loop, demonstrating how an agent can use a language model to reason about tasks, decide on actions, and interpret observations in a structured environment.
---
## **4. OpenAI Client Initialization**
- Initializes an `OpenAI` client, which is used to interact with the language model (e.g., GPT-4).
- This client is utilized within the `ReactEnvironment` state handlers to generate thoughts and actions.
---
## **5. Example Usage Function (`example_usage`)**
**Purpose**: Demonstrates how to instantiate and use the `ReactEnvironment` to perform a task.
### **Process**:
- **Initialization**:
- Creates an instance of `ReactEnvironment` with a specific `task_description` and `available_actions`.
- Defines an `observation_formatter` to process observations.
- **ReAct Loop**:
- Retrieves the initial context from the environment.
- Enters a loop that continues until the agent reaches the `FINISHED` state.
- In each iteration, the agent:
- Thinks: Generates a thought about what to do next.
- Acts: Decides on and performs an action based on the thought.
- Observes: Processes the observation resulting from the action.
- This loop simulates how an agent can use reasoning and acting in tandem to accomplish a task.
---
## **Summary**
- **Overall Functionality**:
- The code provides a framework for building Gymnasium environments that are controlled by state machines.
- It integrates a language model to enable an agent to perform complex reasoning and decision-making tasks.
- Through the `ReactEnvironment`, it demonstrates how to implement the ReAct paradigm, where an agent cycles through thinking, acting, and observing to achieve a goal.
- **Key Concepts**:
- **State Machines**: Managing the environment's states and transitions systematically.
- **Functional Programming**: Using immutable contexts and pure functions to handle state changes.
- **Language Model Integration**: Leveraging GPT-4 to generate thoughts and actions based on prompts.
- **ReAct Paradigm**: Combining reasoning and acting in a loop to solve tasks.
- **Use Cases**:
- This framework can be used to build intelligent agents that require complex decision-making capabilities.
- It's suitable for tasks where reasoning about actions and their consequences is crucial.
---
## **Detailed Walkthrough**
Let's delve deeper into some parts of the code for a better understanding.
### **StateMachineEnv Class**
- **Action Handling in `step` Method**:
- The `step` method can handle both simple actions (just an integer) and parameterized actions (a tuple of action index and parameters).
- It retrieves the corresponding trigger from the `actions` dictionary.
- If the action definition is a dictionary, it merges any default parameters with provided ones.
- It attempts to execute the trigger method, which corresponds to a transition in the state machine.
- **Callbacks in Transitions**:
- The `_add_transitions` method incorporates callbacks for conditions, before actions, and after actions.
- This allows for custom logic to be executed during state transitions, enabling more complex behaviors.
### **ReactEnvironment Class**
- **Prompt Creation**:
- **Thinking Prompt**:
- Incorporates the task description, available actions, and the agent's history into the prompt.
- Encourages the language model to generate reasoning about what to do next.
- **Acting Prompt**:
- Focuses on extracting an action from the agent's thought.
- Asks the language model to return just the action name, ensuring that the output is actionable.
- **Handling Invalid Actions**:
- If the language model suggests an action that is not in the list of available actions, the agent is redirected back to the thinking state.
- This allows the agent to reconsider and generate a valid action.
- **Observation Formatting**:
- The `observation_formatter` is a callable provided during initialization.
- It defines how observations are processed and presented to the agent.
### **Interacting with the OpenAI API**
- The code uses the `openai` library to interact with the language model.
- For each prompt, it sends a message to the model and retrieves the response.
- The model's parameters, such as `temperature`, can be adjusted to control the randomness of the outputs.
### **Example Task in `example_usage`**
- **Task**: "Find a red book on the shelf."
- **Available Actions**: `"search[entity]"`, `"lookup[string]"`, `"finish[answer]"`.
- **Process**:
- The agent starts in the `THINKING` state and generates thoughts about how to accomplish the task.
- It then decides on an action (e.g., `search[entity]`) based on its thought.
- After acting, it observes the result and processes it.
- This loop continues until the agent finishes the task.
---
## **Conclusion**
The code provides a comprehensive framework for creating environments where agents can perform reasoning and actions in a structured manner. By integrating a powerful language model, it enables the development of intelligent agents capable of handling complex tasks that require both thought and action. The use of state machines and functional programming principles ensures that the environment is robust, flexible, and maintainable.
---
Feel free to ask if you have any questions or need further clarification on any part of the code!
To complete and make the provided code fully functional, several key areas need to be addressed. The code outlines a framework for creating a state machine-driven environment that integrates with an OpenAI language model to perform tasks using the ReAct (Reasoning and Acting) paradigm. However, there are placeholders, incomplete implementations, and potential issues that need resolution. Below is a detailed explanation of what is needed to finish the code:
---
## **1. Implement Placeholder Methods in `StateMachineEnv`**
The `StateMachineEnv` class has several methods that are currently placeholders or return default values. To make the environment functional, you need to implement these methods based on the specific requirements of your environment.
### **Methods to Implement:**
- **`_update_observation(self)`**:
- **Current State**: The method is empty (`pass`).
- **What to Do**:
- Update `self._observation` based on the current state of the environment.
- Define how the observation reflects the state machine's state.
- For example, if your observations are numerical representations of states, you might set `self._observation = self.states.index(self.state)`.
- **`compute_reward(self, action: int) -> float`**:
- **Current State**: Returns `0.0` by default.
- **What to Do**:
- Implement logic to compute the reward based on the current state and the action taken.
- Define what constitutes a positive or negative reward in your environment.
- For example, you might return `1.0` when the agent reaches a goal state and `-0.1` for each step to encourage efficiency.
- **`is_terminated(self) -> bool`**:
- **Current State**: Returns `False` by default.
- **What to Do**:
- Implement logic to determine when an episode should terminate.
- Define conditions under which the environment signals that the task is complete.
- For example, return `True` when the agent reaches a final state or exceeds a maximum number of steps.
### **Why This is Needed:**
- **Functionality**: Without these implementations, the environment won't provide meaningful observations, rewards, or termination signals, which are essential for any agent interacting with it.
- **Integration with Agents**: Agents rely on observations to decide actions, rewards to learn policies, and termination signals to reset environments.
---
## **2. Fix Import Statements and Module References**
There are inconsistencies and potential errors in the import statements and how classes are referenced.
### **Issues to Address:**
- **Duplicate Imports**: The code imports the same modules multiple times (e.g., `from typing import List, Dict, Any, Optional, Tuple, Callable, Union` is imported twice).
- **Solution**: Remove duplicate import statements to clean up the code.
- **Incorrect Module References**:
- **Problematic Import**:
```python
from src.state_machine_env import StateMachineEnv
```
- This suggests that `StateMachineEnv` is located in a module named `src.state_machine_env`, but in your code, it's defined in the same script.
- **Solution**:
- Remove this import statement.
- Ensure that when you instantiate `StateMachineEnv`, you reference it directly without importing it from another module.
### **Why This is Needed:**
- **Code Integrity**: Incorrect imports can lead to `ModuleNotFoundError` or `ImportError`.
- **Maintainability**: Clean and correct imports make the code easier to read and maintain.
---
## **3. Correct Usage of the OpenAI API**
The code currently initializes and uses the OpenAI API in a way that may not align with the actual API.
### **Issues to Address:**
- **Initialization of OpenAI Client**:
- **Current Code**:
```python
import openai
from openai import OpenAI
# ...
client = OpenAI()
```
- **Problems**:
- There is no `OpenAI` class in the `openai` module to instantiate.
- The correct way is to use the `openai` module directly after setting the API key.
- **Usage of the API**:
- **Current Code**:
```python
response = client.chat.completions.create(
model="gpt-4",
temperature=0.7,
messages=[{"role": "user", "content": prompt}],
)
```
- The method chain `client.chat.completions.create` does not match the actual OpenAI API methods.
### **Solutions:**
- **Set the OpenAI API Key**:
- Before making any API calls, you need to set your OpenAI API key:
```python
import openai
openai.api_key = 'your-api-key-here'
```
- **Security Note**: Do not hardcode your API key in the code. Instead, use environment variables or a configuration file.
- **Use Correct API Methods**:
- **Chat Completion**:
```python
response = openai.ChatCompletion.create(
model="gpt-4",
temperature=0.7,
messages=[{"role": "user", "content": prompt}],
)
```
- **Accessing the Response**:
```python
thought = response['choices'][0]['message']['content']
```
- **Remove Incorrect Imports and Client Initialization**:
- Remove `from openai import OpenAI` and `client = OpenAI()` from your code.
### **Why This is Needed:**
- **Functionality**: Incorrect usage of the API will lead to runtime errors.
- **Compliance**: Using the API as intended ensures compliance with OpenAI's policies and helps avoid issues like exceeding rate limits or encountering unexpected errors.
---
## **4. Define a Valid `observation_space`**
In the `FunctionalStateEnvironment` initialization, `observation_space` is set to `None`, which is invalid.
### **Issue:**
- **Invalid Observation Space**:
```python
self._state_machine = StateMachineEnv(
states=states,
actions=actions,
initial_state=initial_state,
transitions=transitions,
observation_space=None, # Define based on your needs
)
```
### **Solution:**
- **Define an Appropriate Observation Space**:
- If your observations are discrete states, you might use:
```python
observation_space=gym.spaces.Discrete(len(states))
```
- If your observations are more complex, define the space accordingly using Gymnasium's space definitions.
### **Why This is Needed:**
- **Compliance with Gymnasium API**: The `observation_space` must be a valid `gym.Space` instance.
- **Agent Compatibility**: Agents interacting with the environment rely on the observation space to understand the format and range of observations they will receive.
---
## **5. Ensure All Variables and Methods Are Properly Defined**
Some variables or methods might be missing or incorrectly used in the code.
### **Issues to Address:**
- **Undefined Variables**:
- **Example**: In the `ReactEnvironment`, `context.observation` is used, but it may not be properly initialized or updated.
- **Method Signatures**:
- Ensure that methods like `_handle_thinking`, `_handle_acting`, etc., have the correct parameters and return types.
### **Solutions:**
- **Initialize All Necessary Variables**:
- Ensure that `context.observation` is set before it's used.
- Initialize any variables in the `__init__` method or relevant state handlers.
- **Check Method Definitions**:
- Verify that all methods have the correct parameters and are called with the appropriate arguments.
### **Why This is Needed:**
- **Prevent Runtime Errors**: Undefined variables or incorrect method calls can cause the program to crash.
- **Code Correctness**: Proper variable initialization and method definitions are essential for the code to function as intended.
---
## **6. Implement the `observation_formatter` Function**
In the `ReactEnvironment`, the `observation_formatter` function is used but not fully defined.
### **Issue:**
- **Incomplete Definition**:
- In `example_usage`, `observation_formatter` is defined as:
```python
observation_formatter=lambda action, obs: f"Observed: {obs}",
```
- However, `obs` may not have meaningful data since the `_update_observation` method is not implemented.
### **Solution:**
- **Implement Observation Updates**:
- In `StateMachineEnv`'s `_update_observation` method, set `self._observation` to meaningful data.
- Ensure that `obs` passed to `observation_formatter` contains relevant information.
- **Define `observation_formatter` Appropriately**:
- Customize the formatter to process observations in a way that makes sense for your application.
### **Why This is Needed:**
- **Meaningful Observations**: The agent relies on observations to inform its next steps.
- **Correct Functioning of State Handlers**: The observation is used in the `_handle_observing` method to update the context.
---
## **7. Address the Transition from `ACTING` to `OBSERVING`**
In the `ReactEnvironment`, after acting, the agent is supposed to observe the outcome. The code needs to ensure that the observation reflects the action taken.
### **Issues to Address:**
- **Observation Generation**:
- How does the environment generate observations based on actions?
- Currently, there's no logic that connects the action to an observation.
### **Solutions:**
- **Implement Action Effects**:
- In the `_handle_acting` method, after determining the action, simulate the effect of the action on the environment.
- Update the environment's state or data accordingly.
- **Generate Observations**:
- In the `_update_observation` method of `StateMachineEnv`, generate observations based on the current state and the effects of the action.
- Ensure that when `self.get_observation()` is called, it returns the updated observation.
### **Why This is Needed:**
- **Causal Consistency**: Actions should have effects that are reflected in subsequent observations.
- **Agent Learning**: The agent learns from the consequences of its actions, which is essential for intelligent behavior.
---
## **8. Implement Logic for Completing the Task**
The code currently lacks a mechanism for the agent to determine when it has completed the task.
### **Issues to Address:**
- **Termination Condition**:
- The agent should transition to the `FINISHED` state when the task is completed.
- Currently, there is no logic that defines when and how the `finish` trigger is invoked.
### **Solutions:**
- **Define a `finish` Trigger**:
- In the `ReactEnvironment`, add logic to determine when the agent should finish.
- For example, in the `_handle_observing` method, check if the observation indicates task completion.
- If the task is complete, invoke the `finish` trigger:
```python
if task_is_complete:
self._state_machine.finish()
```
- **Update `is_terminated` Method**:
- In `StateMachineEnv`, update the `is_terminated` method to return `True` when `self.state == 'FINISHED'`.
### **Why This is Needed:**
- **Task Completion**: Without a mechanism to finish the task, the agent could loop indefinitely.
- **Environment Compliance**: The environment should signal when the episode is over so that agents can reset or update their learning process.
---
## **9. Test and Debug the Code**
After making the above changes, thoroughly test the code to ensure it functions as intended.
### **Steps to Take:**
- **Run the `example_usage` Function**:
- Execute the function to see how the agent behaves.
- Observe the outputs and any errors that occur.
- **Debug Any Issues**:
- Use debugging tools or print statements to trace the flow of execution.
- Check for exceptions or incorrect behavior.
- **Validate the Agent's Decisions**:
- Ensure that the thoughts, actions, and observations make sense in the context of the task.
- Adjust parameters like the language model's temperature to influence the agent's behavior.
### **Why This is Needed:**
- **Functionality Verification**: Testing confirms that the code changes have achieved the desired effect.
- **Quality Assurance**: Debugging helps identify and fix issues that could cause the agent to behave unexpectedly.
---
## **10. Consider Enhancing the Agent's Capabilities**
To make the agent more robust and effective, you might consider additional improvements.
### **Potential Enhancements:**
- **Error Handling**:
- Implement mechanisms to handle invalid or unexpected responses from the language model.
- For example, if the agent's thought doesn't lead to a valid action, prompt it to rethink.
- **Memory and Context Management**:
- Limit the history size to prevent prompts from becoming too long.
- Use summarization to condense the history if necessary.
- **Optimization of API Calls**:
- Use techniques to minimize API usage and reduce costs, such as caching responses or adjusting the level of detail in prompts.
- **Customization of Prompts**:
- Fine-tune prompts to elicit better responses from the language model.
- Experiment with different prompt styles or instructions.
### **Why This is Needed:**
- **Agent Performance**: Enhancements can lead to better decision-making and task completion rates.
- **Resource Management**: Optimizing API usage can reduce costs and improve efficiency.
---
## **Summary**
To finish the code:
1. **Implement Placeholder Methods**: Provide concrete implementations for `_update_observation`, `compute_reward`, and `is_terminated` in `StateMachineEnv`.
2. **Fix Imports**: Remove unnecessary or incorrect import statements and ensure all modules are correctly referenced.
3. **Correct OpenAI API Usage**: Initialize the API properly and use the correct methods for interacting with the language model.
4. **Define `observation_space`**: Set a valid observation space in `FunctionalStateEnvironment` and ensure it's appropriate for your observations.
5. **Ensure Variable and Method Definitions**: Check that all variables are initialized and methods are correctly defined and called.
6. **Implement `observation_formatter`**: Define how observations are formatted and ensure observations are meaningful.
7. **Connect Actions to Observations**: Implement logic to reflect the effects of actions in observations.
8. **Implement Task Completion Logic**: Define how and when the agent recognizes task completion and transitions to the `FINISHED` state.
9. **Test and Debug**: Run the code, test functionality, and debug any issues that arise.
10. **Enhance Agent Capabilities**: Consider improvements to make the agent more robust and efficient.
---
By addressing these points, you will create a functional environment where an agent can use the ReAct paradigm to perform tasks using reasoning and action in a structured, state-driven framework. This will allow you to simulate complex tasks and potentially develop advanced agents capable of sophisticated decision-making processes.
If you need further assistance with any specific part of the code or have questions about implementing these suggestions, feel free to ask!
# Implementation Plan Breakdown
## Phase 1: Core Environment Setup
### 1.1. Base State Machine Implementation
1. **`_update_observation` Method**
- Define observation data structure
- Implement state-to-observation mapping
- Add validation for observation updates
- Handle edge cases (invalid states, transitions)
2. **Reward Computation**
- Define reward structure
```python
def compute_reward(self, action: int) -> float:
rewards = {
'FINISHED': 1.0,
'INVALID_ACTION': -0.1,
'STEP_PENALTY': -0.01
}
# Implementation logic
```
- Add action-specific rewards
- Implement state-dependent rewards
- Add reward scaling/normalization
3. **Termination Conditions**
```python
def is_terminated(self) -> bool:
return (
self.state == 'FINISHED' or
self.steps >= self.max_steps or
self._check_failure_conditions()
)
```
- Define success conditions
- Implement failure conditions
- Add timeout handling
- Include safety termination conditions
4. **State Transition Testing**
- Test valid transitions
- Test invalid transitions
- Verify callback execution
- Test transition side effects
### 1.2. Observation Space Definition
1. **Space Structure**
```python
self.observation_space = gym.spaces.Dict({
'state': gym.spaces.Discrete(len(states)),
'context': gym.spaces.Box(low=0, high=1, shape=(context_dim,)),
'history': gym.spaces.Sequence(...)
})
```
- Define observation dimensions
- Set up bounds and constraints
- Add type validation
2. **Observation Formatting**
- Implement serialization
- Add deserialization
- Handle complex data types
- Implement compression (if needed)
3. **Validation System**
```python
def _validate_observation(self, obs: Any) -> bool:
try:
return self.observation_space.contains(obs)
except Exception as e:
self._handle_validation_error(e)
```
- Add range checks
- Implement type checking
- Add format validation
- Include error handling
## Phase 2: Functional Layer
### 2.1. State Context Implementation
1. **Immutable Context**
```python
@dataclass(frozen=True)
class StateContext:
state: str
data: Dict[str, Any]
observation: Any
metadata: Dict[str, Any]
timestamp: float
```
- Define core attributes
- Add validation rules
- Implement deep copying
- Add serialization support
2. **Data Validation**
```python
class StateContextValidator:
@staticmethod
def validate_data(data: Dict[str, Any]) -> None:
# Validation logic
```
- Add type checking
- Implement value validation
- Add required field checks
- Include format validation
3. **Context Updates**
```python
def update_context(self, updates: Dict[str, Any]) -> StateContext:
new_data = {**self._context.data, **updates}
return StateContext(
state=self._context.state,
data=new_data,
observation=self._context.observation
)
```
- Implement immutable updates
- Add change tracking
- Include validation
- Handle nested updates
### 2.2. State Handler Implementation
1. **Handler Interface**
```python
class StateHandler(Protocol):
def __call__(self, context: StateContext, action: Any) -> StateContext:
...
```
- Define handler protocol
- Add type hints
- Include documentation
- Add error handling
2. **Basic Handlers**
```python
def create_handler(
state: str,
transitions: List[Dict[str, Any]],
callbacks: Dict[str, Callable]
) -> StateHandler:
# Handler creation logic
```
- Implement state transitions
- Add action processing
- Include observation updates
- Add callback support
2. **OpenAI Integration**
- Set up API configuration
- Implement error handling for API calls
- Add rate limiting
- Test API responses
## Phase 3: ReAct Environment
### 3.1. Basic Structure
1. **ReAct States**
```python
class ReactStates:
THINKING = "THINKING"
ACTING = "ACTING"
OBSERVING = "OBSERVING"
FINISHED = "FINISHED"
@classmethod
def all_states(cls) -> List[str]:
return [v for k, v in vars(cls).items()
if not k.startswith('_')]
```
- Define state constants
- Add state validation
- Include state metadata
- Add state transitions
2. **Action Definition**
```python
class ReactActions:
def __init__(self):
self.actions = {
0: {"name": "think", "params": {}},
1: {"name": "act", "params": {"action_type": str}},
2: {"name": "observe", "params": {}}
}
```
- Define action space
- Add parameter validation
- Include action metadata
- Add action constraints
3. **Performance Optimization**
- Add caching for API calls
- Optimize prompt lengths
- Implement history management
- Profile and optimize bottlenecks
### 3.3. Prompt Engineering
1. **Thinking Prompts**
```python
class PromptBuilder:
@staticmethod
def build_thinking_prompt(
context: ReactContext,
task: str,
history: List[Dict[str, Any]]
) -> str:
# Prompt building logic
```
- Define prompt templates
- Add context integration
- Include history formatting
- Add prompt validation
2. **Action Extraction**
```python
class ActionExtractor:
@staticmethod
def extract_action(
thought: str,
available_actions: List[str]
) -> Optional[str]:
# Action extraction logic
```
- Implement parsing logic
- Add validation rules
- Include fallback handling
- Add action normalization
3. **History Management**
```python
class HistoryManager:
def __init__(self, max_entries: int = 10):
self.max_entries = max_entries
self.entries: List[ReactState] = []
def add_entry(self, entry: ReactState) -> None:
# History management logic
```
- Implement history truncation
- Add summarization
- Include relevance scoring
- Add compression
## Phase 6: Enhancement & Extensions
1. **Memory Management**
- Implement history pruning
- Add summarization
- Create context management
- Test memory efficiencyAn AI client and API for WordPress to communicate with any generative AI models of various capabilities using a uniform API. Built on top of the [PHP AI Client](https://github.com/WordPress/php-ai-client), it provides a WordPress-native Prompt Builder, an Admin Settings Screen for credentials, automatic credential wiring, a PSR-compliant HTTP client, and a client-side JavaScript API.
> This file provides instructions for AI agents that read AGENTS.md (GitHub Copilot, Cursor, Windsurf, Cline, Aider, OpenCode, and others).
This document collects ideas and instructions for implementing future improvements. Follow these when adding features or refactoring the code.
> This file must stay **in sync** with `CLAUDE.md`. Whenever you change one, mirror the same change in the other so both tools continue to work correctly.