|
| 1 | +# :simple-owasp: Asset Database |
| 2 | + |
| 3 | +The **Asset DB** is the **PostgreSQL implementation** of the database layer for the OWASP Amass framework. It provides a robust database interaction layer for storing and managing the [Open Asset Model (OAM)](https://github.com/owasp-amass/open-asset-model). While Amass supports multiple database backends (including Neo4j/Bolt , SQLite), the Asset Database specifically implements the PostgreSQL storage layer with optimized schema and query capabilities. |
| 4 | + |
| 5 | +--- |
| 6 | + |
| 7 | +## // Overview |
| 8 | + |
| 9 | +The [Asset Database](https://github.com/owasp-amass/asset-db) is designed to facilitate an ecosystem of scanning tools, allowing the storage of assets and their complex relationships in a structured, queryable format. This enables: |
| 10 | + |
| 11 | +- **Persistent Asset Intel**: Store discovered assets and their relationships for long-term analysis. |
| 12 | +- **Query Complex Relationships**: Use the [Triples Query Language](triples.md) to traverse asset relationships. |
| 13 | +- **Track Asset Evolution**: Monitor changes in your attack surface over time. |
| 14 | +- **Interoperability**: Provide a unified data layer for security tools. |
| 15 | + |
| 16 | +!!! info "Open Asset Model Integration" |
| 17 | + The Asset Database is built around the [Open Asset Model](https://github.com/owasp-amass/open-asset-model), which defines standardized asset types, properties, and relationships. This ensures consistency across different tools and enables comprehensive attack surface mapping beyond just internet infrastructure. |
| 18 | + |
| 19 | +--- |
| 20 | + |
| 21 | +## // Key Features |
| 22 | + |
| 23 | +#### :octicons-database-16: Graph Database: |
| 24 | + |
| 25 | +- **PostgreSQL Backend**: Optimized schema and extensions for PostgreSQL. |
| 26 | +- **Graph-based Storage**: Leverages PostgreSQL's graph capabilities for relationship queries. |
| 27 | +- **Scalable Architecture**: Designed for enterprise environments with large asset inventories. |
| 28 | +- **Triples Query Language**: Advanced graph traversal language built for PostgreSQL. |
| 29 | + |
| 30 | +#### :octicons-graph-16: Complex Mapping: |
| 31 | + |
| 32 | +- **Asset Relationships**: Store and query complex relationships between different asset types. |
| 33 | +- **Triples Query Language**: Powerful graph traversal language for complex queries. |
| 34 | +- **Multi-triple Traversals**: Support for up to 10 triples for complex graph walks. |
| 35 | +- **Temporal Tracking**: Track when relationships were discovered and their confidence levels. |
| 36 | +- **Attribute Filtering**: Filter results by asset and relation attributes. |
| 37 | + |
| 38 | +#### :octicons-tools-16: System Integration: |
| 39 | + |
| 40 | +- **Command Line Interface**: Subcommand querying via `amass assoc`. |
| 41 | +- **Database Interface**: Direct database access for programmatic integration. |
| 42 | +- **Modular Architecture**: Extensible design supporting custom integrations. |
| 43 | +- **Export Capabilities**: Export data in multiple formats for analysis and reporting. |
| 44 | + |
| 45 | +--- |
| 46 | + |
| 47 | +## // Supported Asset Types |
| 48 | + |
| 49 | +The Asset Database supports all asset types defined in the [Open Asset Model](../open_asset_model/index.md). For detailed information about each asset type, see [Assets](../open_asset_model/assets/index.md). |
| 50 | + |
| 51 | +--- |
| 52 | + |
| 53 | +## //Architecture |
| 54 | + |
| 55 | + |
| 56 | +The Asset Database follows a **layered architecture pattern** that provides exceptional flexibility, scalability, and maintainability. This design pattern separates concerns into distinct layers, each with specific responsibilities and clear interfaces between them. |
| 57 | + |
| 58 | +--- |
| 59 | + |
| 60 | +```mermaid |
| 61 | +graph LR |
| 62 | + subgraph "Client Layer" |
| 63 | + direction TB |
| 64 | + A["Amass Core"] |
| 65 | + B["Data Sources"] |
| 66 | + C["Go Library"] |
| 67 | + end |
| 68 | +
|
| 69 | + subgraph "Repository Layer" |
| 70 | + direction TB |
| 71 | + D["Asset Storage"] |
| 72 | + E["Relationship Mgmt"] |
| 73 | + F["Query Interface"] |
| 74 | + end |
| 75 | +
|
| 76 | + subgraph "Database Layer" |
| 77 | + direction TB |
| 78 | + G[(PostgreSQL)] |
| 79 | + H["Schema Mgmt"] |
| 80 | + I["pg_trgm Extension"] |
| 81 | + end |
| 82 | +
|
| 83 | + %% Flows |
| 84 | + A --> D |
| 85 | + B --> D |
| 86 | + C --> D |
| 87 | + D --> E |
| 88 | + E --> F |
| 89 | + F --> G |
| 90 | + G --> H |
| 91 | + G --> I |
| 92 | +``` |
| 93 | +--- |
| 94 | + |
| 95 | +#### :material-console: **Client Layer** >> Interface & Integration |
| 96 | + |
| 97 | +> This layer handles user interactions and data ingestion: |
| 98 | +
|
| 99 | +- **Amass Core**: Primary enumeration engine and discovery framework. |
| 100 | + |
| 101 | +- **Data Sources**: External feeds and tools that populate the database. |
| 102 | + |
| 103 | +- **Go Library**: Programmatic access for custom integrations and automation. |
| 104 | + |
| 105 | +#### :material-spider-web: **Repository Layer** >> Data Abstraction & Logic |
| 106 | + |
| 107 | +> This layer implements the core data operations and query logic: |
| 108 | +
|
| 109 | +- **Asset Storage**: CRUD operations for all asset types (FQDNs, IPs, Organizations, etc.). |
| 110 | + |
| 111 | +- **Relationship Management**: Graph relationship storage and traversal logic. |
| 112 | + |
| 113 | +- **Query Interface**: Triples Query Language implementation and query processing. |
| 114 | + |
| 115 | +- **Abstraction**: Provides a consistent entry point for data operations and removed underlying database complexity. |
| 116 | + |
| 117 | +#### :material-table-column: **Database Layer** >> Data Persistence & Storage |
| 118 | + |
| 119 | +> This layer handles data persistence and optimization: |
| 120 | +
|
| 121 | + - **PostgreSQL**: Primary database with optimized schema for graph relationships. |
| 122 | + |
| 123 | + - **Schema Management**: Table structures and indexing for efficient queries. |
| 124 | + |
| 125 | + - **`pg_trgm`**: Trigram indexing for fuzzy string matching and search. |
| 126 | + |
| 127 | +--- |
| 128 | + |
| 129 | +#### Why This Works Well: |
| 130 | + |
| 131 | +**>> Separation of Concerns** |
| 132 | + |
| 133 | +>Each layer has a specific responsibility: |
| 134 | +
|
| 135 | +- **Client Layer**: User interface and data ingestion |
| 136 | + |
| 137 | +- **Repository Layer**: Data abstraction and business logic |
| 138 | + |
| 139 | +- **Database Layer**: Data persistence and storage |
| 140 | + |
| 141 | +> This separation enables independent development and testing. |
| 142 | +
|
| 143 | +#### **>> Multiple Integration Patterns** |
| 144 | +>The system supports various access methods: |
| 145 | +
|
| 146 | +- **Command Line**: Direct querying via `amass assoc` command |
| 147 | + |
| 148 | +- **Go Library**: Programmatic access for custom applications |
| 149 | + |
| 150 | +- **Database**: Direct SQL access for advanced analytics |
| 151 | + |
| 152 | +#### **>> Database Flexibility** |
| 153 | +>The Repository Layer abstracts database details, enabling: |
| 154 | +
|
| 155 | +- **PostgreSQL**: Primary implementation with graph capabilities. |
| 156 | + |
| 157 | +- **Neo4j**: Graph database for complex relationships |
| 158 | + |
| 159 | +- **SQLite**: Lightweight option for basic deployments |
| 160 | + |
| 161 | +#### **>> Modular Design** |
| 162 | + |
| 163 | +>The layered architecture provides: |
| 164 | +
|
| 165 | +- **Independent Development**: Teams can work on different layers |
| 166 | +- **Clear Interfaces**: Well-defined boundaries between components |
| 167 | +- **Extensible Structure**: Easy to add new features or modify existing ones |
| 168 | + |
| 169 | +#### **>> Maintainability** |
| 170 | +>Clear layer boundaries enable: |
| 171 | +
|
| 172 | +- **Isolated Testing**: Each layer can be tested independently |
| 173 | +- **Problem Isolation**: Issues can be traced to specific layers |
| 174 | +- **Independent Updates**: Changes in one layer don't affect others |
| 175 | + |
| 176 | +--- |
| 177 | + |
| 178 | +This design means you can: |
| 179 | + |
| 180 | +- **Start Simple**: Begin with basic enumeration and add more later |
| 181 | +- **Grow When Needed**: Add more data sources as you need them |
| 182 | +- **Use Your Tools**: Work with your existing security setup |
| 183 | +- **Build Your Own**: Create custom tools if you want to |
| 184 | +- **Keep It Working**: Make changes without breaking everything |
| 185 | + |
| 186 | +--- |
| 187 | + |
| 188 | +## // Common Use Cases |
| 189 | + |
| 190 | +#### >> Security Research: |
| 191 | + |
| 192 | +- **Find Assets**: Discover domains, IPs, and other assets you didn't know about |
| 193 | +- **Track Changes**: See what's new or changed in your target's infrastructure |
| 194 | +- **Map Relationships**: Understand how different assets connect to each other |
| 195 | + |
| 196 | +#### >> Bug Bounty & Penetration Testing: |
| 197 | + |
| 198 | +- **Scope Discovery**: Find all the assets in your target's attack surface |
| 199 | +- **Asset Tracking**: Keep track of what you've found during your research |
| 200 | +- **Relationship Mapping**: See how assets relate to each other for better targeting |
| 201 | + |
| 202 | +#### >> Security Analysis: |
| 203 | + |
| 204 | +- **Asset Inventory**: Build a complete picture of what you're analyzing |
| 205 | +- **Historical Tracking**: See what assets were discovered when |
| 206 | +- **Data Export**: Get your findings out for further analysis |
| 207 | +``` |
| 208 | +
|
| 209 | +
|
0 commit comments