Skip to content

Commit df789d9

Browse files
KN4CK3Rlunny
andauthored
Add Cargo package registry (#21888)
This PR implements a [Cargo registry](https://doc.rust-lang.org/cargo/) to manage Rust packages. This package type was a little bit more complicated because Cargo needs an additional Git repository to store its package index. Screenshots: ![grafik](https://user-images.githubusercontent.com/1666336/203102004-08d812ac-c066-4969-9bda-2fed818554eb.png) ![grafik](https://user-images.githubusercontent.com/1666336/203102141-d9970f14-dca6-4174-b17a-50ba1bd79087.png) ![grafik](https://user-images.githubusercontent.com/1666336/203102244-dc05743b-78b6-4d97-998e-ef76341a978f.png) --------- Co-authored-by: Lunny Xiao <[email protected]>
1 parent 7baeb9c commit df789d9

File tree

35 files changed

+1660
-125
lines changed

35 files changed

+1660
-125
lines changed

custom/conf/app.example.ini

+2
Original file line numberDiff line numberDiff line change
@@ -2458,6 +2458,8 @@ ROUTER = console
24582458
;LIMIT_TOTAL_OWNER_COUNT = -1
24592459
;; Maximum size of packages a single owner can use (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
24602460
;LIMIT_TOTAL_OWNER_SIZE = -1
2461+
;; Maximum size of a Cargo upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
2462+
;LIMIT_SIZE_CARGO = -1
24612463
;; Maximum size of a Composer upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
24622464
;LIMIT_SIZE_COMPOSER = -1
24632465
;; Maximum size of a Conan upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)

docs/content/doc/advanced/config-cheat-sheet.en-us.md

+1
Original file line numberDiff line numberDiff line change
@@ -1213,6 +1213,7 @@ Task queue configuration has been moved to `queue.task`. However, the below conf
12131213
- `CHUNKED_UPLOAD_PATH`: **tmp/package-upload**: Path for chunked uploads. Defaults to `APP_DATA_PATH` + `tmp/package-upload`
12141214
- `LIMIT_TOTAL_OWNER_COUNT`: **-1**: Maximum count of package versions a single owner can have (`-1` means no limits)
12151215
- `LIMIT_TOTAL_OWNER_SIZE`: **-1**: Maximum size of packages a single owner can use (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
1216+
- `LIMIT_SIZE_CARGO`: **-1**: Maximum size of a Cargo upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
12161217
- `LIMIT_SIZE_COMPOSER`: **-1**: Maximum size of a Composer upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
12171218
- `LIMIT_SIZE_CONAN`: **-1**: Maximum size of a Conan upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
12181219
- `LIMIT_SIZE_CONDA`: **-1**: Maximum size of a Conda upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
+109
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
---
2+
date: "2022-11-20T00:00:00+00:00"
3+
title: "Cargo Packages Repository"
4+
slug: "packages/cargo"
5+
draft: false
6+
toc: false
7+
menu:
8+
sidebar:
9+
parent: "packages"
10+
name: "Cargo"
11+
weight: 5
12+
identifier: "cargo"
13+
---
14+
15+
# Cargo Packages Repository
16+
17+
Publish [Cargo](https://doc.rust-lang.org/stable/cargo/) packages for your user or organization.
18+
19+
**Table of Contents**
20+
21+
{{< toc >}}
22+
23+
## Requirements
24+
25+
To work with the Cargo package registry, you need [Rust and Cargo](https://www.rust-lang.org/tools/install).
26+
27+
Cargo stores informations about the available packages in a package index stored in a git repository.
28+
This repository is needed to work with the registry.
29+
The following section describes how to create it.
30+
31+
## Index Repository
32+
33+
Cargo stores informations about the available packages in a package index stored in a git repository.
34+
In Gitea this repository has the special name `_cargo-index`.
35+
After a package was uploaded, its metadata is automatically written to the index.
36+
The content of this repository should not be manually modified.
37+
38+
The user or organization package settings page allows to create the index repository along with the configuration file.
39+
If needed this action will rewrite the configuration file.
40+
This can be useful if for example the Gitea instance domain was changed.
41+
42+
If the case arises where the packages stored in Gitea and the information in the index repository are out of sync, the settings page allows to rebuild the index repository.
43+
This action iterates all packages in the registry and writes their information to the index.
44+
If there are lot of packages this process may take some time.
45+
46+
## Configuring the package registry
47+
48+
To register the package registry the Cargo configuration must be updated.
49+
Add the following text to the configuration file located in the current users home directory (for example `~/.cargo/config.toml`):
50+
51+
```
52+
[registry]
53+
default = "gitea"
54+
55+
[registries.gitea]
56+
index = "https://gitea.example.com/{owner}/_cargo-index.git"
57+
58+
[net]
59+
git-fetch-with-cli = true
60+
```
61+
62+
| Parameter | Description |
63+
| --------- | ----------- |
64+
| `owner` | The owner of the package. |
65+
66+
If the registry is private or you want to publish new packages, you have to configure your credentials.
67+
Add the credentials section to the credentials file located in the current users home directory (for example `~/.cargo/credentials.toml`):
68+
69+
```
70+
[registries.gitea]
71+
token = "Bearer {token}"
72+
```
73+
74+
| Parameter | Description |
75+
| --------- | ----------- |
76+
| `token` | Your [personal access token]({{< relref "doc/developers/api-usage.en-us.md#authentication" >}}) |
77+
78+
## Publish a package
79+
80+
Publish a package by running the following command in your project:
81+
82+
```shell
83+
cargo publish
84+
```
85+
86+
You cannot publish a package if a package of the same name and version already exists. You must delete the existing package first.
87+
88+
## Install a package
89+
90+
To install a package from the package registry, execute the following command:
91+
92+
```shell
93+
cargo add {package_name}
94+
```
95+
96+
| Parameter | Description |
97+
| -------------- | ----------- |
98+
| `package_name` | The package name. |
99+
100+
## Supported commands
101+
102+
```
103+
cargo publish
104+
cargo add
105+
cargo install
106+
cargo yank
107+
cargo unyank
108+
cargo search
109+
```

docs/content/doc/packages/overview.en-us.md

+1
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ The following package managers are currently supported:
2626

2727
| Name | Language | Package client |
2828
| ---- | -------- | -------------- |
29+
| [Cargo]({{< relref "doc/packages/cargo.en-us.md" >}}) | Rust | `cargo` |
2930
| [Composer]({{< relref "doc/packages/composer.en-us.md" >}}) | PHP | `composer` |
3031
| [Conan]({{< relref "doc/packages/conan.en-us.md" >}}) | C++ | `conan` |
3132
| [Conda]({{< relref "doc/packages/conda.en-us.md" >}}) | - | `conda` |

models/packages/descriptor.go

+3
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import (
1111
repo_model "code.gitea.io/gitea/models/repo"
1212
user_model "code.gitea.io/gitea/models/user"
1313
"code.gitea.io/gitea/modules/json"
14+
"code.gitea.io/gitea/modules/packages/cargo"
1415
"code.gitea.io/gitea/modules/packages/composer"
1516
"code.gitea.io/gitea/modules/packages/conan"
1617
"code.gitea.io/gitea/modules/packages/conda"
@@ -129,6 +130,8 @@ func GetPackageDescriptor(ctx context.Context, pv *PackageVersion) (*PackageDesc
129130

130131
var metadata interface{}
131132
switch p.Type {
133+
case TypeCargo:
134+
metadata = &cargo.Metadata{}
132135
case TypeComposer:
133136
metadata = &composer.Metadata{}
134137
case TypeConan:

models/packages/package.go

+6
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ type Type string
3030

3131
// List of supported packages
3232
const (
33+
TypeCargo Type = "cargo"
3334
TypeComposer Type = "composer"
3435
TypeConan Type = "conan"
3536
TypeConda Type = "conda"
@@ -46,6 +47,7 @@ const (
4647
)
4748

4849
var TypeList = []Type{
50+
TypeCargo,
4951
TypeComposer,
5052
TypeConan,
5153
TypeConda,
@@ -64,6 +66,8 @@ var TypeList = []Type{
6466
// Name gets the name of the package type
6567
func (pt Type) Name() string {
6668
switch pt {
69+
case TypeCargo:
70+
return "Cargo"
6771
case TypeComposer:
6872
return "Composer"
6973
case TypeConan:
@@ -97,6 +101,8 @@ func (pt Type) Name() string {
97101
// SVGName gets the name of the package type svg image
98102
func (pt Type) SVGName() string {
99103
switch pt {
104+
case TypeCargo:
105+
return "gitea-cargo"
100106
case TypeComposer:
101107
return "gitea-composer"
102108
case TypeConan:

models/packages/package_property.go

+6
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,12 @@ func GetPropertiesByName(ctx context.Context, refType PropertyType, refID int64,
5858
return pps, db.GetEngine(ctx).Where("ref_type = ? AND ref_id = ? AND name = ?", refType, refID, name).Find(&pps)
5959
}
6060

61+
// UpdateProperty updates a property
62+
func UpdateProperty(ctx context.Context, pp *PackageProperty) error {
63+
_, err := db.GetEngine(ctx).ID(pp.ID).Update(pp)
64+
return err
65+
}
66+
6167
// DeleteAllProperties deletes all properties of a ref
6268
func DeleteAllProperties(ctx context.Context, refType PropertyType, refID int64) error {
6369
_, err := db.GetEngine(ctx).Where("ref_type = ? AND ref_id = ?", refType, refID).Delete(&PackageProperty{})

modules/packages/cargo/parser.go

+169
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
// Copyright 2022 The Gitea Authors. All rights reserved.
2+
// SPDX-License-Identifier: MIT
3+
4+
package cargo
5+
6+
import (
7+
"encoding/binary"
8+
"errors"
9+
"io"
10+
"regexp"
11+
12+
"code.gitea.io/gitea/modules/json"
13+
"code.gitea.io/gitea/modules/validation"
14+
15+
"github.com/hashicorp/go-version"
16+
)
17+
18+
const PropertyYanked = "cargo.yanked"
19+
20+
var (
21+
ErrInvalidName = errors.New("package name is invalid")
22+
ErrInvalidVersion = errors.New("package version is invalid")
23+
)
24+
25+
// Package represents a Cargo package
26+
type Package struct {
27+
Name string
28+
Version string
29+
Metadata *Metadata
30+
Content io.Reader
31+
ContentSize int64
32+
}
33+
34+
// Metadata represents the metadata of a Cargo package
35+
type Metadata struct {
36+
Dependencies []*Dependency `json:"dependencies,omitempty"`
37+
Features map[string][]string `json:"features,omitempty"`
38+
Authors []string `json:"authors,omitempty"`
39+
Description string `json:"description,omitempty"`
40+
DocumentationURL string `json:"documentation_url,omitempty"`
41+
ProjectURL string `json:"project_url,omitempty"`
42+
Readme string `json:"readme,omitempty"`
43+
Keywords []string `json:"keywords,omitempty"`
44+
Categories []string `json:"categories,omitempty"`
45+
License string `json:"license,omitempty"`
46+
RepositoryURL string `json:"repository_url,omitempty"`
47+
Links string `json:"links,omitempty"`
48+
}
49+
50+
type Dependency struct {
51+
Name string `json:"name"`
52+
Req string `json:"req"`
53+
Features []string `json:"features"`
54+
Optional bool `json:"optional"`
55+
DefaultFeatures bool `json:"default_features"`
56+
Target *string `json:"target"`
57+
Kind string `json:"kind"`
58+
Registry *string `json:"registry"`
59+
Package *string `json:"package"`
60+
}
61+
62+
var nameMatch = regexp.MustCompile(`\A[a-zA-Z][a-zA-Z0-9-_]{0,63}\z`)
63+
64+
// ParsePackage reads the metadata and content of a package
65+
func ParsePackage(r io.Reader) (*Package, error) {
66+
var size uint32
67+
if err := binary.Read(r, binary.LittleEndian, &size); err != nil {
68+
return nil, err
69+
}
70+
71+
p, err := parsePackage(io.LimitReader(r, int64(size)))
72+
if err != nil {
73+
return nil, err
74+
}
75+
76+
if err := binary.Read(r, binary.LittleEndian, &size); err != nil {
77+
return nil, err
78+
}
79+
80+
p.Content = io.LimitReader(r, int64(size))
81+
p.ContentSize = int64(size)
82+
83+
return p, nil
84+
}
85+
86+
func parsePackage(r io.Reader) (*Package, error) {
87+
var meta struct {
88+
Name string `json:"name"`
89+
Vers string `json:"vers"`
90+
Deps []struct {
91+
Name string `json:"name"`
92+
VersionReq string `json:"version_req"`
93+
Features []string `json:"features"`
94+
Optional bool `json:"optional"`
95+
DefaultFeatures bool `json:"default_features"`
96+
Target *string `json:"target"`
97+
Kind string `json:"kind"`
98+
Registry *string `json:"registry"`
99+
ExplicitNameInToml string `json:"explicit_name_in_toml"`
100+
} `json:"deps"`
101+
Features map[string][]string `json:"features"`
102+
Authors []string `json:"authors"`
103+
Description string `json:"description"`
104+
Documentation string `json:"documentation"`
105+
Homepage string `json:"homepage"`
106+
Readme string `json:"readme"`
107+
ReadmeFile string `json:"readme_file"`
108+
Keywords []string `json:"keywords"`
109+
Categories []string `json:"categories"`
110+
License string `json:"license"`
111+
LicenseFile string `json:"license_file"`
112+
Repository string `json:"repository"`
113+
Links string `json:"links"`
114+
}
115+
if err := json.NewDecoder(r).Decode(&meta); err != nil {
116+
return nil, err
117+
}
118+
119+
if !nameMatch.MatchString(meta.Name) {
120+
return nil, ErrInvalidName
121+
}
122+
123+
if _, err := version.NewSemver(meta.Vers); err != nil {
124+
return nil, ErrInvalidVersion
125+
}
126+
127+
if !validation.IsValidURL(meta.Homepage) {
128+
meta.Homepage = ""
129+
}
130+
if !validation.IsValidURL(meta.Documentation) {
131+
meta.Documentation = ""
132+
}
133+
if !validation.IsValidURL(meta.Repository) {
134+
meta.Repository = ""
135+
}
136+
137+
dependencies := make([]*Dependency, 0, len(meta.Deps))
138+
for _, dep := range meta.Deps {
139+
dependencies = append(dependencies, &Dependency{
140+
Name: dep.Name,
141+
Req: dep.VersionReq,
142+
Features: dep.Features,
143+
Optional: dep.Optional,
144+
DefaultFeatures: dep.DefaultFeatures,
145+
Target: dep.Target,
146+
Kind: dep.Kind,
147+
Registry: dep.Registry,
148+
})
149+
}
150+
151+
return &Package{
152+
Name: meta.Name,
153+
Version: meta.Vers,
154+
Metadata: &Metadata{
155+
Dependencies: dependencies,
156+
Features: meta.Features,
157+
Authors: meta.Authors,
158+
Description: meta.Description,
159+
DocumentationURL: meta.Documentation,
160+
ProjectURL: meta.Homepage,
161+
Readme: meta.Readme,
162+
Keywords: meta.Keywords,
163+
Categories: meta.Categories,
164+
License: meta.License,
165+
RepositoryURL: meta.Repository,
166+
Links: meta.Links,
167+
},
168+
}, nil
169+
}

0 commit comments

Comments
 (0)