Merge pull request #14 from SalesforceAIResearch/zhiwei/dev

Zhiwei/dev

Author: JimSalesforce

ROADMAP.md

@@ -3,6 +3,8 @@
 ### MCP Server 🖥️
 - ✅ Python stdio server support
 - ✅ node.js stdio server support
+- ✅ http mcp server support
+- 🔲 connecting mcp servers with json file as a standard way
 
 ### MCP Client 🤖
 - ✅ Stdio client implementation
@@ -19,20 +21,14 @@
 ### Evaluation 📊
 - ✅ Implement core evaluation metrics (accuracy, latency)
 - ✅ Create automated testing framework
+- 🔲 Automatic Deep Evaluating 
+- 🔲 Evaluating the implementation of MCP server
 
 ### Data Pipeline 🔄
 - ✅ Design unified data schema for all benchmarks
 - ✅ Implement data preprocessing tools
 - ✅ Add support for multiple data formats
 
-### Benchmarks 🧪
-- ✅ Airbnb MCP benchmark
-- ✅ Healthcare MCP benchmark
-- ✅ yahoo finance MCP benchmark
-- ✅ Sports benchmark
-- ✅ travel_assistant benchmark
-- ✅ File System benchmark
-
 ### LLM Provider 🧠
 - ✅ OpenAI API integration (used for data generation and testing)
 - ✅ local vllm-based model 
@@ -43,4 +39,21 @@
 - ✅ Data converter
 - ✅ Model evaluator
 - ✅ Report generator
-- ✅ Auto end-to-end evaluation
+- ✅ Auto end-to-end evaluation
+
+### Front-end 🎨
+- ✅ React application setup with TypeScript
+- ✅ Core navigation and routing
+- ✅ MCP server configuration interface
+- ✅ Chat client for MCP interactions
+- ✅ Task generation and verification UI
+- ✅ Model evaluation dashboard
+- ✅ Results and analytics pages
+- ✅ Data management interfaces
+- 🔲 Unifying the model config for all the pages and sharing the same component
+- 🔲 Saving any existing model config as a config file and support load it again
+
+## Issues
+- Evluating multiple models does not working
+- Analyze feature does not support not generating AI report
+- Judge Rubrics select not generate report