Development Guide
Learn how to extend and customize ExtendedLM
Project Structure
Understanding ExtendedLM's codebase organization.
extendedlm/
├── app/ # Next.js app directory
│ ├── (auth)/ # Authentication pages
│ │ ├── login/
│ │ └── signup/
│ ├── api/ # API routes
│ │ ├── chat/ # Chat endpoints
│ │ ├── workflows/ # Workflow endpoints
│ │ ├── rag/ # RAG endpoints
│ │ ├── mcp/ # MCP endpoints
│ │ └── admin/ # Admin endpoints
│ ├── components/ # React components
│ │ ├── Chat/ # Chat interface
│ │ ├── Sidebar/ # Sidebar components
│ │ ├── Artifacts/ # Artifact renderers
│ │ ├── Workflows/ # Workflow editor
│ │ └── Settings/ # Settings panels
│ ├── lib/ # Shared libraries
│ │ ├── agents/ # Agent implementations
│ │ ├── rag/ # RAG system
│ │ ├── mcp/ # MCP client
│ │ ├── tools/ # LLM tools
│ │ └── utils/ # Utilities
│ ├── hooks/ # React hooks
│ ├── types/ # TypeScript types
│ ├── config/ # Configuration files
│ └── globals.css # Global styles
├── gateway/ # Rust Gateway
│ ├── src/
│ │ ├── main.rs
│ │ ├── llama.rs
│ │ ├── routes/
│ │ └── models/
│ └── Cargo.toml
├── mate/ # Python Mate
│ ├── app/
│ │ ├── main.py
│ │ ├── browser.py
│ │ ├── files.py
│ │ └── shell.py
│ └── requirements.txt
├── mcp-servers/ # MCP servers
│ ├── cal2prompt/
│ ├── switchbot/
│ └── custom/
├── prisma/ # Database schema
│ └── schema.prisma
├── public/ # Static assets
├── scripts/ # Build/deployment scripts
├── tests/ # Test files
│ ├── unit/
│ ├── integration/
│ └── e2e/
├── .env.local # Environment variables
├── package.json
├── tsconfig.json
└── next.config.js
Key Directories
- app/api/ - Next.js API routes for all endpoints
- app/components/ - React components organized by feature
- app/lib/agents/ - Agent implementations
- app/lib/rag/ - RAG system core logic
- gateway/ - Rust-based local LLM gateway
- lmmate/ - Python FastAPI for computer use
- mcp-servers/ - Model Context Protocol servers
Development Setup
Setting up your local development environment.
Prerequisites
- Node.js 20+
- npm or yarn
- PostgreSQL 16+ with pgvector
- Rust 1.75+ (for Gateway)
- Python 3.11+ (for Mate)
- Docker (optional, for containerized services)
Installation
# Clone repository
git clone https://github.com/yourusername/extendedlm.git
cd extendedlm
# Install dependencies
npm install
# Set up environment
cp .env.example .env.local
# Edit .env.local with your API keys and config
# Set up database
createdb extendedlm
npx prisma migrate dev
# Generate Prisma client
npx prisma generate
# Build Gateway (optional, for local LLM)
cd gateway
cargo build --release
cd ..
# Set up Mate (optional, for computer use)
cd mate
python -m venv venv
source venv/bin/activate # On Windows: venvScriptsctivate
pip install -r requirements.txt
cd ..
# Start development server
npm run dev
Development Commands
# Start Next.js dev server
npm run dev
# Build for production
npm run build
# Start production server
npm start
# Run tests
npm test
# Run tests in watch mode
npm run test:watch
# Run E2E tests
npm run test:e2e
# Lint code
npm run lint
# Format code
npm run format
# Type check
npm run type-check
# Database commands
npx prisma studio # Open Prisma Studio
npx prisma migrate dev # Create migration
npx prisma migrate reset # Reset database
Creating Custom Agents
Extend ExtendedLM with your own specialized agents.
Agent Structure
// File: app/lib/agents/my-custom-agent.ts
import { Agent, AgentConfig, Message } from '@/types/agent';
import { generateCompletion } from '@/lib/llm';
export const myCustomAgent: Agent = {
id: 'my-custom',
name: 'My Custom Agent',
description: 'Description of what this agent does',
icon: '🤖',
color: '#3b82f6',
// System prompt
systemPrompt: `You are a specialized assistant that...`,
// Tools available to this agent
tools: [
{
name: 'custom_tool',
description: 'Performs a custom operation',
parameters: {
type: 'object',
properties: {
input: {
type: 'string',
description: 'Input to process',
},
},
required: ['input'],
},
},
],
// Tool implementations
async executeTool(toolName: string, args: any) {
switch (toolName) {
case 'custom_tool':
return await this.customTool(args.input);
default:
throw new Error(`Unknown tool: ${toolName}`);
}
},
async customTool(input: string) {
// Implement your custom logic
const result = await processInput(input);
return result;
},
// Process user message
async processMessage(config: AgentConfig) {
const { messages, model, temperature } = config;
const response = await generateCompletion({
model,
messages: [
{ role: 'system', content: this.systemPrompt },
...messages,
],
temperature,
tools: this.tools,
onToolCall: async (toolCall) => {
const result = await this.executeTool(
toolCall.name,
toolCall.arguments
);
return result;
},
});
return response;
},
// Decide if this agent should handle the query
async shouldHandle(message: string): Promise {
// Implement logic to determine if this agent is appropriate
const keywords = ['custom', 'special', 'unique'];
return keywords.some((kw) => message.toLowerCase().includes(kw));
},
// Transfer to another agent if needed
async transferTo(): Promise {
// Return agent ID to transfer to, or null to stay
return null;
},
};
export default myCustomAgent;
Register Agent
// File: app/lib/agents/index.ts
import standardAgent from './standard-agent';
import ragAgent from './rag-agent';
import myCustomAgent from './my-custom-agent';
export const agents = {
standard: standardAgent,
rag: ragAgent,
'my-custom': myCustomAgent,
};
export function getAgent(id: string) {
return agents[id];
}
export function getAllAgents() {
return Object.values(agents);
}
Use in UI
// File: app/components/Settings/AgentSelector.tsx
import { getAllAgents } from '@/lib/agents';
export function AgentSelector() {
const agents = getAllAgents();
return (
<select>
{agents.map((agent) => (
<option key={agent.id} value={agent.id}>
{agent.icon} {agent.name}
</option>
))}
</select>
);
}
Creating Custom Tools
Add new tools for LLMs to use.
Tool Definition
// File: app/lib/tools/my-tool.ts
import { Tool, ToolDefinition } from '@/types/tool';
export const myTool: Tool = {
definition: {
name: 'fetch_weather',
description: 'Fetches current weather for a location',
parameters: {
type: 'object',
properties: {
location: {
type: 'string',
description: 'City name or coordinates',
},
units: {
type: 'string',
enum: ['metric', 'imperial'],
description: 'Temperature units',
default: 'metric',
},
},
required: ['location'],
},
},
async execute(args: { location: string; units?: string }) {
const { location, units = 'metric' } = args;
// Call weather API
const response = await fetch(
`https://api.openweathermap.org/data/2.5/weather?q=${location}&units=${units}&appid=${process.env.OPENWEATHER_API_KEY}`
);
const data = await response.json();
return {
location: data.name,
temperature: data.main.temp,
description: data.weather[0].description,
humidity: data.main.humidity,
wind_speed: data.wind.speed,
};
},
};
export default myTool;
Register Tool
// File: app/lib/tools/index.ts
import { myTool } from './my-tool';
import { searchWeb } from './search-web';
import { runCode } from './run-code';
export const tools = {
fetch_weather: myTool,
search_web: searchWeb,
run_code: runCode,
};
export function getTool(name: string) {
return tools[name];
}
export function getAllTools() {
return Object.values(tools);
}
export function getToolDefinitions() {
return getAllTools().map((tool) => tool.definition);
}
Use in Agent
// Add tool to agent
import { myTool } from '@/lib/tools/my-tool';
export const weatherAgent = {
id: 'weather',
name: 'Weather Agent',
tools: [myTool.definition],
async executeTool(name: string, args: any) {
if (name === 'fetch_weather') {
return await myTool.execute(args);
}
},
};
Extending RAG System
Customize the Retrieval-Augmented Generation pipeline.
Custom Chunking Strategy
// File: app/lib/rag/chunkers/my-chunker.ts
import { Chunker, Chunk } from '@/types/rag';
export class MyCustomChunker implements Chunker {
constructor(
private chunkSize: number = 1000,
private chunkOverlap: number = 200
) {}
async chunk(text: string, metadata: any): Promise<Chunk[]> {
const chunks: Chunk[] = [];
// Implement your custom chunking logic
// Example: Split by semantic boundaries
const sentences = text.match(/[^.!?]+[.!?]+/g) || [];
let currentChunk = '';
let chunkIndex = 0;
for (const sentence of sentences) {
if (currentChunk.length + sentence.length > this.chunkSize) {
chunks.push({
id: `chunk_${chunkIndex}`,
content: currentChunk.trim(),
metadata: {
...metadata,
chunk_index: chunkIndex,
sentence_count: currentChunk.split(/[.!?]+/).length,
},
});
// Overlap: keep last sentence
const lastSentence = currentChunk.split(/[.!?]+/).slice(-1)[0];
currentChunk = lastSentence + ' ' + sentence;
chunkIndex++;
} else {
currentChunk += ' ' + sentence;
}
}
// Add final chunk
if (currentChunk.trim()) {
chunks.push({
id: `chunk_${chunkIndex}`,
content: currentChunk.trim(),
metadata: {
...metadata,
chunk_index: chunkIndex,
},
});
}
return chunks;
}
}
export default MyCustomChunker;
Custom Embedding Model
// File: app/lib/rag/embedders/my-embedder.ts
import { Embedder } from '@/types/rag';
export class MyCustomEmbedder implements Embedder {
async embed(texts: string[]): Promise<number[][]> {
// Call your custom embedding API
const response = await fetch('https://my-embedding-api.com/embed', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ texts }),
});
const { embeddings } = await response.json();
return embeddings;
}
async embedQuery(query: string): Promise<number[]> {
const embeddings = await this.embed([query]);
return embeddings[0];
}
getDimensions(): number {
return 1536; // Your embedding dimension
}
}
export default MyCustomEmbedder;
Custom Retriever
// File: app/lib/rag/retrievers/my-retriever.ts
import { Retriever, RetrievalResult } from '@/types/rag';
export class MyCustomRetriever implements Retriever {
async retrieve(
query: string,
k: number = 5,
threshold: number = 0.7
): Promise<RetrievalResult[]> {
// Generate query embedding
const queryEmbedding = await this.embedQuery(query);
// Custom retrieval logic
// Example: Combine vector search with keyword search
// Vector search
const vectorResults = await this.vectorSearch(queryEmbedding, k * 2);
// Keyword search
const keywordResults = await this.keywordSearch(query, k * 2);
// Merge and rerank
const merged = this.mergeResults(vectorResults, keywordResults);
const reranked = await this.rerank(query, merged);
// Filter by threshold and limit
return reranked
.filter((result) => result.score >= threshold)
.slice(0, k);
}
private async vectorSearch(embedding: number[], k: number) {
// Implement vector similarity search
const results = await db.query(`
SELECT id, content, metadata,
1 - (embedding <=> $1) AS similarity
FROM document_chunks
ORDER BY embedding <=> $1
LIMIT $2
`, [embedding, k]);
return results.rows;
}
private async keywordSearch(query: string, k: number) {
// Implement full-text search
const results = await db.query(`
SELECT id, content, metadata,
ts_rank(search_vector, plainto_tsquery($1)) AS rank
FROM document_chunks
WHERE search_vector @@ plainto_tsquery($1)
ORDER BY rank DESC
LIMIT $2
`, [query, k]);
return results.rows;
}
private mergeResults(vectorResults: any[], keywordResults: any[]) {
// Implement result fusion (e.g., Reciprocal Rank Fusion)
const combined = new Map();
vectorResults.forEach((result, index) => {
combined.set(result.id, {
...result,
score: result.similarity,
vector_rank: index + 1,
});
});
keywordResults.forEach((result, index) => {
if (combined.has(result.id)) {
const existing = combined.get(result.id);
existing.keyword_rank = index + 1;
existing.score = (existing.score + result.rank) / 2;
} else {
combined.set(result.id, {
...result,
score: result.rank,
keyword_rank: index + 1,
});
}
});
return Array.from(combined.values());
}
private async rerank(query: string, results: any[]) {
// Implement cross-encoder reranking
const pairs = results.map((r) => [query, r.content]);
const scores = await this.crossEncoderScore(pairs);
return results
.map((result, index) => ({
...result,
score: scores[index],
}))
.sort((a, b) => b.score - a.score);
}
}
export default MyCustomRetriever;
Creating MCP Servers
Build custom Model Context Protocol servers for new integrations.
Python MCP Server
# File: mcp-servers/my-server/server.py
from mcp.server import Server
from mcp.types import Tool, Resource
import asyncio
app = Server("my-server")
# Define tools
@app.list_tools()
async def list_tools():
return [
Tool(
name="my_tool",
description="Description of what this tool does",
inputSchema={
"type": "object",
"properties": {
"param1": {
"type": "string",
"description": "First parameter"
},
"param2": {
"type": "number",
"description": "Second parameter"
}
},
"required": ["param1"]
}
)
]
# Implement tool
@app.call_tool()
async def call_tool(name: str, arguments: dict):
if name == "my_tool":
param1 = arguments["param1"]
param2 = arguments.get("param2", 0)
# Your tool logic here
result = f"Processed {param1} with {param2}"
return {
"content": [
{
"type": "text",
"text": result
}
]
}
# Define resources
@app.list_resources()
async def list_resources():
return [
Resource(
uri="my-server://data",
name="My Data",
mimeType="application/json"
)
]
# Serve resources
@app.read_resource()
async def read_resource(uri: str):
if uri == "my-server://data":
data = {"example": "data"}
return {
"contents": [
{
"uri": uri,
"mimeType": "application/json",
"text": json.dumps(data)
}
]
}
if __name__ == "__main__":
asyncio.run(app.run())
TypeScript MCP Server
// File: mcp-servers/my-server/src/index.ts
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
import {
ListToolsRequestSchema,
CallToolRequestSchema,
} from '@modelcontextprotocol/sdk/types.js';
const server = new Server(
{
name: 'my-server',
version: '1.0.0',
},
{
capabilities: {
tools: {},
resources: {},
},
}
);
// List tools
server.setRequestHandler(ListToolsRequestSchema, async () => {
return {
tools: [
{
name: 'my_tool',
description: 'Description of what this tool does',
inputSchema: {
type: 'object',
properties: {
param1: {
type: 'string',
description: 'First parameter',
},
param2: {
type: 'number',
description: 'Second parameter',
},
},
required: ['param1'],
},
},
],
};
});
// Call tool
server.setRequestHandler(CallToolRequestSchema, async (request) => {
const { name, arguments: args } = request.params;
if (name === 'my_tool') {
const { param1, param2 = 0 } = args as {
param1: string;
param2?: number;
};
// Your tool logic here
const result = `Processed ${param1} with ${param2}`;
return {
content: [
{
type: 'text',
text: result,
},
],
};
}
throw new Error(`Unknown tool: ${name}`);
});
// Start server
async function main() {
const transport = new StdioServerTransport();
await server.connect(transport);
}
main().catch(console.error);
Register MCP Server
// File: mcp-config.json
{
"mcpServers": {
"my-server": {
"command": "python",
"args": ["-m", "mcp_servers.my_server"],
"env": {
"API_KEY": "${MY_API_KEY}"
},
"enabled": true,
"description": "My custom MCP server"
}
}
}
Testing
Write tests for your custom code.
Unit Tests
// File: tests/unit/agents/my-custom-agent.test.ts
import { describe, it, expect, vi } from 'vitest';
import myCustomAgent from '@/lib/agents/my-custom-agent';
describe('My Custom Agent', () => {
it('should handle appropriate messages', async () => {
const message = 'This is a custom query';
const shouldHandle = await myCustomAgent.shouldHandle(message);
expect(shouldHandle).toBe(true);
});
it('should execute custom tool', async () => {
const result = await myCustomAgent.executeTool('custom_tool', {
input: 'test',
});
expect(result).toBeDefined();
});
it('should process message correctly', async () => {
const messages = [
{ role: 'user', content: 'Test message' },
];
const response = await myCustomAgent.processMessage({
messages,
model: 'gpt-4o',
temperature: 0.7,
});
expect(response).toBeDefined();
expect(response.content).toBeTruthy();
});
});
Integration Tests
// File: tests/integration/api/chat.test.ts
import { describe, it, expect } from 'vitest';
import { createMocks } from 'node-mocks-http';
import { POST as chatHandler } from '@/app/api/chat/route';
describe('Chat API', () => {
it('should create conversation and send message', async () => {
const { req, res } = createMocks({
method: 'POST',
body: {
conversation_id: 'test-conv',
message: 'Hello',
model: 'gpt-4o',
},
headers: {
'Authorization': 'Bearer test-api-key',
},
});
const response = await chatHandler(req);
const data = await response.json();
expect(data.success).toBe(true);
expect(data.data.content).toBeTruthy();
});
});
E2E Tests
// File: tests/e2e/chat-flow.spec.ts
import { test, expect } from '@playwright/test';
test.describe('Chat Flow', () => {
test('should create conversation and send message', async ({ page }) => {
await page.goto('http://localhost:3000');
// Login
await page.fill('input[name="email"]', 'test@example.com');
await page.fill('input[name="password"]', 'password');
await page.click('button[type="submit"]');
// Wait for chat interface
await page.waitForSelector('[data-testid="message-input"]');
// Send message
await page.fill('[data-testid="message-input"]', 'Hello, AI!');
await page.click('[data-testid="send-button"]');
// Wait for response
await page.waitForSelector('[data-testid="assistant-message"]');
// Verify response
const response = await page.textContent('[data-testid="assistant-message"]');
expect(response).toBeTruthy();
});
});
Running Tests
# Run all tests
npm test
# Run specific test file
npm test my-custom-agent.test.ts
# Run tests in watch mode
npm run test:watch
# Run E2E tests
npm run test:e2e
# Generate coverage report
npm run test:coverage
Code Style & Best Practices
Follow these guidelines for consistent code.
TypeScript Best Practices
- Use TypeScript strict mode
- Define interfaces for all data structures
- Avoid
anytype - use proper typing - Use async/await instead of promises
- Handle errors with try/catch
- Use meaningful variable names
- Write JSDoc comments for public APIs
React Best Practices
- Use functional components with hooks
- Memoize expensive computations with
useMemo - Memoize callbacks with
useCallback - Split large components into smaller ones
- Use custom hooks for reusable logic
- Avoid prop drilling - use context or state management
- Use Suspense for lazy loading
Formatting
# Format code with Prettier
npm run format
# Check formatting
npm run format:check
# Lint code
npm run lint
# Fix linting issues
npm run lint:fix
ESLint Configuration
// .eslintrc.json
{
"extends": [
"next/core-web-vitals",
"plugin:@typescript-eslint/recommended",
"prettier"
],
"rules": {
"@typescript-eslint/no-unused-vars": "error",
"@typescript-eslint/no-explicit-any": "warn",
"react-hooks/rules-of-hooks": "error",
"react-hooks/exhaustive-deps": "warn"
}
}
Contributing
Guidelines for contributing to ExtendedLM.
Fork and Clone
# Fork repository on GitHub
# Then clone your fork
git clone https://github.com/YOUR_USERNAME/extendedlm.git
cd extendedlm
# Add upstream remote
git remote add upstream https://github.com/original/extendedlm.git
Create Feature Branch
# Create branch from main
git checkout -b feature/my-new-feature
# Or for bug fix
git checkout -b fix/issue-123
Make Changes
- Write code following style guidelines
- Add tests for new functionality
- Update documentation if needed
- Ensure all tests pass
- Run linter and formatter
Commit Changes
# Stage changes
git add .
# Commit with descriptive message
git commit -m "feat: add custom agent support"
# Commit message format:
# feat: new feature
# fix: bug fix
# docs: documentation changes
# style: code style changes (formatting)
# refactor: code refactoring
# test: adding tests
# chore: maintenance tasks
Push and Create PR
# Push to your fork
git push origin feature/my-new-feature
# Create Pull Request on GitHub
# Provide clear description of changes
# Link related issues
PR Guidelines
- One feature/fix per PR
- Clear and descriptive title
- Detailed description of changes
- Screenshots/videos for UI changes
- Link to related issues
- Request review from maintainers
- Address review feedback promptly
Code Review Process
- Automated checks (CI/CD) must pass
- At least one maintainer approval required
- Address review comments
- Squash commits if requested
- PR merged by maintainer
CI/CD Pipeline
Automated testing and deployment workflows.
GitHub Actions Workflow
# File: .github/workflows/ci.yml
name: CI
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: pgvector/pgvector:pg16
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run linter
run: npm run lint
- name: Run type check
run: npm run type-check
- name: Run tests
run: npm test
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test
- name: Build
run: npm run build
e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: '20'
- name: Install dependencies
run: npm ci
- name: Install Playwright
run: npx playwright install --with-deps
- name: Run E2E tests
run: npm run test:e2e
Deployment Workflow
# File: .github/workflows/deploy.yml
name: Deploy
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Deploy to Vercel
uses: amondnet/vercel-action@v20
with:
vercel-token: ${{ secrets.VERCEL_TOKEN }}
vercel-org-id: ${{ secrets.VERCEL_ORG_ID }}
vercel-project-id: ${{ secrets.VERCEL_PROJECT_ID }}
vercel-args: '--prod'
Documentation
Writing and maintaining documentation.
Code Documentation
/**
* Retrieves relevant document chunks for a query using hybrid search.
*
* @param query - The search query
* @param options - Retrieval options
* @param options.k - Number of chunks to retrieve (default: 5)
* @param options.threshold - Minimum similarity threshold (default: 0.7)
* @param options.rerank - Whether to rerank results (default: true)
* @returns Array of retrieved chunks with scores
*
* @example
* ```typescript
* const chunks = await retrieveChunks('machine learning', {
* k: 10,
* threshold: 0.8,
* rerank: true
* });
* ```
*/
export async function retrieveChunks(
query: string,
options: RetrievalOptions = {}
): Promise<RetrievalResult[]> {
// Implementation...
}
API Documentation
Update API reference docs when adding new endpoints:
- Endpoint path and method
- Request/response examples
- Parameter descriptions
- Error responses
- Usage examples in multiple languages
User Documentation
Update user-facing docs for new features:
- Feature overview
- Step-by-step guides
- Screenshots/videos
- Common use cases
- Troubleshooting tips
Release Process
Steps for creating a new release.
Version Numbering
Follow Semantic Versioning (SemVer): MAJOR.MINOR.PATCH
- MAJOR: Breaking changes
- MINOR: New features (backward compatible)
- PATCH: Bug fixes
Release Steps
# 1. Update version in package.json
npm version minor # or major/patch
# 2. Update CHANGELOG.md
# Add release notes for new version
# 3. Commit version bump
git add package.json CHANGELOG.md
git commit -m "chore: bump version to 1.2.0"
# 4. Create tag
git tag -a v1.2.0 -m "Release v1.2.0"
# 5. Push to GitHub
git push origin main --tags
# 6. Create GitHub Release
# Go to GitHub → Releases → Create new release
# Select tag, add release notes, publish
# 7. Deploy to production
# Automated via CI/CD or manual deployment
Changelog Format
# Changelog
## [1.2.0] - 2025-01-15
### Added
- Custom agent support
- New RAG retrieval strategies
- MCP server management UI
### Changed
- Improved workflow execution performance
- Updated UI components
### Fixed
- Fixed race condition in message streaming
- Resolved memory leak in vector search
### Breaking Changes
- Changed API response format for /api/chat endpoint