+1
日增长
-8
周增长
-23
月增长
14,650
stars
637
forks
118
watchers
79
prs
133
issues
0
releases
README
FauxPilot This is an attempt to build a locally hosted alternative to GitHub Copilot. It uses the SalesForce CodeGen models inside of NVIDIA's Triton Inference Server with the FasterTransformer backend. Prerequisites You'll need: Docker docker compose = 1.28 An NVIDIA GPU with Compute Capability = 6.0 and enough VRAM to run the model you want. nvidia-docker curl and zstd for downloading and unpacking the models. Note that the VRAM requirements listed by setup.sh are total -- if you have multiple GPUs, you can split the model across them. So, if you have two NVIDIA RTX 3080 GPUs, you should be able to run the 6B model by putting half on each GPU. Support and Warranty lmao Okay, fine, we now have some minimal information on the wiki and a discussion forum where you can ask questions. Still no formal support or warranty though! Setup This section describes how to install a Fauxpilot server and clients. Setting up a FauxPilot Server Run the setup script to choose a model to use. This will download the model from Huggingface/Moyix in GPT-J format and then convert it for use with FasterTransformer. Please refer to How to set-up a FauxPilot server. Client configuration for FauxPilot We offer some ways to connect to FauxPilot Server. For example, you can create a client by how to open the Openai API, Copilot Plugin, REST API. Please refer to How to set-up a client. Terminology API: Application Programming Interface CC: Compute Capability CUDA: Compute Unified Device Architecture FT: Faster Transformer JSON: JavaScript Object Notation gRPC: Remote Procedure call by Google GPT-J: A transformer model trained using Ben Wang's Mesh Transformer JAX REST: REpresentational State Transfer
Top issues
#1VSCode plugin29 comments · 2022-08-03#53Apple silicon support?13 comments · 2022-09-16#150Triton cannot find config.pbtxt12 comments · 2023-02-16#20[FT][ERROR] CUDA runtime error: operation not supported12 comments · 2022-08-12#12Any way of porting this to Colab?10 comments · 2022-08-06#202CodeGen2 compatibility9 comments · 2023-05-11#62How to optimize CodeGen for my code before launching FauxPilot9 comments · 2022-09-21#198Python backend - fauxpilot-copilot_proxy failed to connect to all addresses8 comments · 2023-04-29
Discussions
#250Llama30 comments · 2024-07-04#247Suggestions for a Chat UI frontend0 comments · 2024-03-12#245How add and using deepseek-coder LLM?0 comments · 2024-02-21#244is this Repository Active0 comments · 2024-02-16#239How do we get this working on Apple Sillicon?0 comments · 2023-12-24#238Can this project be used with OpenAI's GPT-4 or gpt-3.5-turbo-instruct?0 comments · 2023-12-14#233Remote deployment0 comments · 2023-09-07#232how to use?0 comments · 2023-09-05