docs_to_kg

Build Knowledge Graph from Markdown Documents, with OpenAI, Neo4j and CocoIndex

In this example, we

• Extract relationships from Markdown documents.
• Build a knowledge graph from the relationships.

Please give Cocoindex on Github a star to support us if you like our work. Thank you so much with a warm coconut hug 🥥🤗.

Prerequisite

Before running the example, you need to:

• Install Postgres if you don't have one.
• Install Neo4j if you don't have one.
• Install / configure LLM API. In this example we use OpenAI. You need to configure OpenAI API key before running the example. Alternatively, you can also follow the comments in source code to switch to Ollama, which runs LLM model locally, and get it ready following this guide.

Run

Build the index

Install dependencies:

pip install -e .

Setup:

python main.py cocoindex setup

Update index:

python main.py cocoindex update

Browse the knowledge graph

After the knowledge graph is build, you can explore the knowledge graph you built in Neo4j Browser.

For the dev enviroment, you can connect neo4j browser using credentials:

• username: neo4j
• password: cocoindex which is pre-configured in the our docker compose config.yaml.

You can open it at http://localhost:7474, and run the following Cypher query to get all relationships:

MATCH p=()-->() RETURN p

CocoInsight

CocoInsight is a tool to help you understand your data pipeline and data index. CocoInsight is in Early Access now (Free) 😊 You found us! A quick 3 minute video tutorial about CocoInsight: Watch on YouTube.

Run CocoInsight to understand your RAG data pipeline:

python main.py cocoindex server -c https://cocoindex.io

Then open the CocoInsight UI at https://cocoindex.io/cocoinsight. It connects to your local CocoIndex server with zero data retention.

You can view the pipeline flow and the data preview in the CocoInsight UI: