Inpainting Pipeline Implementation on ai-runner Bounty [$850]

[JJassonn69]

Overview

We’re excited to expand the capabilities of our AI-network by implementing an Inpainting pipeline! This addition will unlock advanced image manipulation features, boosting the functionality and versatility of our AI tools. 🔥

We invite the community to contribute to this exciting initiative by helping integrate the Inpainting pipeline into the AI-worker component of the network. Your contribution will enhance image editing capabilities on the Livepeer network, pushing the boundaries of what's possible with (decentralised) AI-driven media. 🚀

Required Skillset

• Familiarity with FastAPI.
• Strong Python programming skills.
• Familiar with docker and Hugging Face 🤗

Bounty Requirements

1. Implementation: Develop a working /in-paint route and pipeline in the AI-worker repository. This pipeline should be accessible on docker port 8600.
2. Functionality: The pipeline must accept an initial image, image mask and text input as prompt and return modified image as result. Also optional parameters such as negative_prompt, blur_factor, strength, guidance_scale should be tuneable according to the inpaining model used.

Scope Exclusions

• This bounty does NOT cover the complete end-to-end implementation of this pipeline on the go-livepeer side, including payment logic and job routing. These aspects will be addressed by the AI SPE team or in a future bounty.

Implementation Tips

To learn how to create a new pipeline, refer to the Hugging Face pipeline documentation. You can also explore the following pull requests to see how other pipelines were implemented:

• Segment Anything 2 Pipeline Implementation
• Audio to Text Pipeline Implementation

In some cases, you may not be able to integrate the new pipeline directly into the regular AI Runner Docker container due to dependency conflicts or missing packages. If this occurs, you can follow the approach outlined in this example to create a custom container for the pipeline. This approach uses the regular AI runner as the base while keeping the base container lean.

Additionally, keep the following best practices in mind:

• Leverage Existing Work: Explore the Hugging Face documentation on inpainting, such as this guide and this project. These resources can provide useful insights and a solid foundation for your implementation.

• Use Developer Documentation: Be sure to consult our developer documentation for both the worker and runner. These resources can help you speed up your development by offering tips on mocking pipelines and enabling direct debugging.

• Generate the OpenAPI Spec: Run the runner/gen_openapi.py script to create an updated OpenAPI specification for your pipeline.

• Generate Go-Livepeer Bindings: In the main repository folder, run the make command to generate the necessary bindings, ensuring seamless integration with the go-livepeer repository.

How to Apply

1. Express Your Interest: Comment on this issue to indicate your interest and explain why you're the ideal candidate for the task.
2. Wait for Review: Our team will review expressions of interest and select the best candidate.
3. Get Assigned: If selected, we'll assign the GitHub issue to you.
4. Start Working: Dive into your task! If you need assistance or guidance, comment on the issue or join the discussions in the #developer-lounge channel on our Discord server.
5. Submit Your Work: Create a pull request in the relevant repository and request a review.
6. Notify Us: Comment on this GitHub issue when your pull request is ready for review.
7. Receive Your Bounty: We'll arrange the bounty payment once your pull request is approved.
8. Gain Recognition: Your valuable contributions will be showcased in our project's changelog.

Thank you for your interest in contributing to our project! 💛

Warning

Please wait for the issue to be assigned to you before starting work. To prevent duplication of effort, submissions for unassigned issues will not be accepted.

[nishantkluhera]

hello!, i'm interested in taking on this bounty

[rickstaa]

Hi @nishantkluhera, thank you for your patience. We've been busy finalizing our public launch at Token2049. Amazing you want to contribute to our open-source system 🚀. I’ve now assigned you to the bounty. If you have any questions, feel free to reach out.

[rickstaa]

@nishantkluhera how is the bounty going? Is there anything we can do to help?

[nishantkluhera]

Hey @rickstaa thanks for checking in!! Everything is progressing well!, have set up the initial structure and currently implementing the inpainting pipeline, will try to deliver a PR by end of week.

I’ll keep you posted on the progress, and will definitely reach out if I need any help (prolly will with testing). Thanks again for the support! 🚀

[rickstaa]

@nishantkluhera amazing thanks for the update! Happy to help.

[rickstaa]

Also, I came across some exciting news— the Transformers library will add support for a new model called ProPainter, which excels at both inpainting and outpainting. You can check it out here: ProPainter on GitHub.

[rickstaa]

Hi @nishantkluhera, apologies for the delay in getting back to you. I just reviewed your submitted solution, and we’re ready to process the payment for your work. Ultimately, we decided to incorporate your contribution into a larger pull request, but we will ensure you are properly credited for your efforts.

Could you please join our Discord server and ping me (rickstaa) in the Builders Lounge (see the image below)? Once there, we can finalize the details of your compensation.

Thank you for your understanding, and we appreciate your contribution 🚀!

[nishantkluhera]

@rickstaa
verification code: 40d52bb3-fef9-47cc-85f6-42e2c44e7c42

[rickstaa]

Completed in livepeer/ai-runner#250 and paid out.