Skip to content

High performance FUSE filesystem for AI workloads with S3 compatible backends

License

Apache-2.0, Unknown licenses found

Licenses found

Apache-2.0
LICENSE
Unknown
LICENSE.geesefs
Notifications You must be signed in to change notification settings

tigrisdata/tigrisfs

unittests xfstests cluster-test

TigrisFS is a high-performance FUSE-based file system for S3-compatible object storage written in Go.

Overview

TigrisFS allows you to mount an S3 or compatible object store bucket as a local file system.

TigrisFS is based on GeeseFS, which is a fork of Goofys. GeeseFS focused on solving performance problems which FUSE file systems based on S3 typically have, especially with small files and metadata operations. It solves these problems by using aggressive parallelism and asynchrony.

The goal of TigrisFS is to further improve on performance and reliability especially in distributed cluster setup.

In the first release we focused on improving security and reliability of the code base:

  • Improved security by:
    • removing bundled, outdated AWS SDK with critical vulnerabilities.
    • upgrading dependencies to fix security vulnerabilities.
  • Improved reliability by:
    • fixing all race conditions found by race detector, which is now enabled by default in tests.
    • fixing all linter issues and enabling linting by default in CI.
    • running more extensive tests and enabling them by default in CI.

Tigris specific features

When mounted with the Tigris backend TigrisFS supports:

  • POSIX permissions, special files, symbolic links.
  • Auto-preload content of small files on directory list in single request.
  • Allows to auto prefetch directory data to the region on list.

Installation

Recommended: One-line install

curl -sSL https://raw.githubusercontent.com/tigrisdata/tigrisfs/refs/heads/main/install.sh | bash

Prebuilt DEB and RPM packages

  • Download the latest release: DEB, RPM.

  • Install the package:

    • Debian-based systems:
      dpkg -i tigrisfs_1.2.1_linux_amd64.deb
    • RPM-based systems:
      rpm -i tigrisfs_1.2.1_linux_amd64.rpm
  • Configure credentials TigrisFS can use credentials from different sources:

    • Standard AWS credentials files ~/.aws/credentials and ~/.aws/config. Use aws configure to set them up and export AWS_PROFILE environment variable to use a specific profile.
    • Environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY.
    • TigrisFS credentials in /etc/default/tigrisfs or mount specific credentials in /etc/default/tigrisfs-<bucket>. See docs for more details.
  • Mount the bucket

    • as current user
      systemctl --user start tigrisfs@<bucket>
      The bucket is mounted at $HOME/mnt/tigris/<bucket>.
    • as root
      systemctl start tigrisfs@<bucket>
      The bucket is mounted at /mnt/tigris/<bucket>.

Binary install

  • Download and unpack the latest release:
    • MacOS ARM64
        curl -L https://github.com/tigrisdata/tigrisfs/releases/download/v1.2.1/tigrisfs_1.2.1_darwin_arm64.tar.gz | sudo tar -xz -C /usr/local/bin
      
  • Configuration is the same as for the DEB and RPM packages above.
  • Mount the bucket:
    • as current user
      /usr/local/bin/tigrisfs <bucket> $HOME/mnt/tigrisfs/<bucket>
    • as root
      sudo /usr/local/bin/tigrisfs <bucket> /mnt/tigrisfs/<bucket>

License

Licensed under the Apache License, Version 2.0

See LICENSE and AUTHORS

About

High performance FUSE filesystem for AI workloads with S3 compatible backends

Resources

License

Apache-2.0, Unknown licenses found

Licenses found

Apache-2.0
LICENSE
Unknown
LICENSE.geesefs

Code of conduct

Contributing

Stars

Watchers

Forks

Packages

No packages published

Contributors 5

Languages