PROJECTS

llm-loader

Setup exllama/llama.cpp loader/API, chat interface Revisit https://github.com/oobabooga/text-generation-webui w/ exllama Easily grab and test models

hfdownloader -m TheBloke/orca_mini_13B-GPTQ -s ./

llm-experiments

• LangChain
  • local: https://www.reddit.com/r/LocalLLaMA/comments/14fqfdu/how_to_use_langchain_with_exllama/
• Vector DB eval

llm-tracker

• Bookstack Setup
• FastAPI API
  • ingest of existing datasets, APIs https://www.reddit.com/r/LocalLLaMA/wiki/models/

lm-evaluation-harness-harness

• Dockerfile
• API Wrappers for exllama and llama.cpp
  • Adapt https://github.com/iaalm/llama-api-server
  • Add Basaran support? https://github.com/hyperonym/basaran
• lm-evaluation-harness-big-refactor
• Output

Quant Test

• lm-eval-h-h
• perplexity

Voice Assistant

• Whisper
• LLM
• Coqui TTS
• https://github.com/yacineMTB/talk
• https://github.com/atisharma/llama_farm

code assistant

See:

• https://aider.chat/
• https://github.com/morph-labs/rift
• Models
  • StarCoder
    • https://arxiv.org/pdf/2305.06161.pdf
  • WizardCoder
    • https://arxiv.org/pdf/2306.08568.pdf
  • phi-1
    • https://arxiv.org/pdf/2306.11644.pdf
• Running
  • koboldcpp https://github.com/LostRuins/koboldcpp/releases/tag/v1.32

PKMS

Summarizer

• Summarize twitter lists, news, etc
• https://github.com/StanGirard/quivr
• https://github.com/openai/openai-cookbook/blob/main/examples/third_party_examples/financial_document_analysis_with_llamaindex.ipynb
• https://twitter.com/mayowaoshin/status/1674462598052433920
• https://github.com/mayooear/private-chatbot-mpt30b-langchain
• https://www.youtube.com/watch?v=CVd2XHcctJo
• https://gist.github.com/hwchase17/c7b8ea57d75bc340a23913cca8e1668f
• https://github.com/camelot-dev/camelot

Local File Indexer

• https://github.com/PromtEngineer/localGPT
• https://github.com/imartinez/privateGPT
• https://github.com/StanGirard/quivr
• https://github.com/jerryjliu/llama_index
• https://github.com/marella/chatdocs

Install and Extend existing software

• Files
• Notes
• Papers
• Photos

Fine Tuning

• https://github.com/AI4Finance-Foundation/FinGPT
• https://erichartford.com/uncensored-models
• https://github.com/artidoro/qlora/blob/main/README.md
• https://erichartford.com/meet-samantha
• https://www.reddit.com/r/LocalLLaMA/comments/13tz14v/how_to_qlora_33b_model_on_a_gpu_with_24gb_of_vram/

https://www.reddit.com/r/LocalLLaMA/comments/14ttt4s/a_simple_repo_for_finetuning_llms_with_both_gptq/ https://github.com/taprosoft/llm_finetuning/tree/main https://github.com/taprosoft/llm_finetuning/blob/main/benchmark/README.md

Tutorials https://github.com/SamurAIGPT/LlamaIndex-course/tree/main https://dev.to/wasp/build-your-own-personal-twitter-agent-with-langchain-32n3