laloop

We put generative AI into production, on your infrastructure.

laloop designs and runs self-hosted RAG and LLM systems. Below is the assistant that answers Se Liga na SKY participants on WhatsApp: embeddings, vector search, a reranker and a local model, with every answer traceable to its source.

Talk about a project See the architecture
Se Liga Supportonline

    Worker trace

    1. querywaiting for a message
    2. embed
    3. pgvector
    4. rerank
    5. threshold
    6. answer

    Conversation translated from Brazilian Portuguese; scores are illustrative. Parameters match the production code.

    Selected work

    The assistant above is one case. Four more, each with its architecture, the decisions that mattered and where it stands today.

    Client work · ML + fraudWho buys parts next month, and which invoice is a fraudPurchase propensity, next best offer and preventive fraud scoring for a manufacturer's dealer network. BigQuery, TensorFlow, XGBoost.5signals blended into one propensity scoreAgentic SRE · LangGraphFrom a vague alert to an evidence-backed hypothesisA human-in-the-loop agent that gathers logs, metrics, releases and flag changes for every alert and writes the triage report.45→8minutes median time-to-context in productionSRE tooling · MCPRunbooks an agent can follow without improvisingA versioned catalogue of SRE and GCP skills, investigation patterns and scripts, served to every agent through one MCP server on Cloudflare Workers.30skills: 25 general, 5 for incident investigationSide project · Sports analyticsTraining analysis a coach can defendLoad, fitness and race predictions for a triathlon squad. Every finding comes from a rule with a cited source; the local LLM only narrates.9rule-based analysis skills, each traced to its sources

    Architecture

    Five containers in one docker-compose. The API only enqueues; the worker serialises access to the GPU. Knowledge lives in a separate repository and is reindexed without rebuilding the image.

    Fig. 01laloop assistant architecture
    laloop assistant architecture The channel sends the message to the FastAPI API, which enqueues it in Redis. The worker consumes the queue, reads the session history from Postgres and runs build_context: BGE-M3 embedding, HNSW search in pgvector, reranking with BGE-reranker-v2-m3 and a threshold. Above the threshold it calls Qwen2.5 14B through Ollama on the GPU; below it, it returns a fixed answer without calling the LLM. In parallel, markdown skills and documents converted by docling are chunked and indexed into pgvector, which is also exposed through an MCP server. ChannelWhatsApp · web FastAPIPOST /chat → 202 job_id Redis 7queue · 1h TTL Workerone job at a time on GPU Postgres 16sessions · 4 msgs · 30 min GET /chat/status · polling 1–2 s build_context() · chat.py BGE-M3embedding 1024d pgvector · HNSW5 skills · 20 chunks BGE-reranker-v2-m3cross-encoder → 1 · 3 Thresholdskill ≥ .65 · doc ≥ .55 Ollama · Qwen2.5 14BNVIDIA GPU Fixed answerLLM is not called yes no Content and ingestion · outside the image, mounted as a volume seliga-rag-contentskills/*/SKILL.md DocumentsPDF · DOCX · PPTX → docling Ingestionchunks 800/120 → BGE-M3 MCP serversame index for Claude, Cursor

    Two-stage retrieval

    Vector search buys cheap recall; the cross-encoder makes the call. Skills and documents run on separate tracks with their own cut-offs and thresholds.

    Stage Skills Documents
    Vector search (cosine, HNSW) 5 candidates 20 chunks
    Cross-encoder reranker 1 kept 3 kept
    Minimum post-rerank score (sigmoid) 0.65 0.55
    What gets embedded name + description 800-character chunk, 120 overlap

    Context budget 12,000 characters in the system prompt; a skill is cut in half if it exceeds that.

    History last 4 messages per session_id, in Postgres, expiring after 30 minutes idle.

    Query current message concatenated with the user's previous message.

    Output sources: [{type, name, chunk, score}] on every answer.

    Decisions that came from production

    Threshold at 0.55, not 0.50

    The reranker returns sigmoid(logit). When it sees neither relevance nor contradiction the logit sits near zero and the score near 0.50. With a 0.50 cut-off, "what is the capital of Brazil?" got through with irrelevant context and the model made things up. At 0.55 it falls into the deterministic path.

    Refusal decided in code

    If no skill or document clears the threshold, grounded=False and the answer is a fixed string. The LLM is never called: the guarantee doesn't depend on the model obeying the prompt, and off-topic questions don't burn GPU time.

    Query augmentation with the previous turn

    "It says my password is wrong but I can't remember it" isn't similar enough to any skill on its own. Concatenated with "I can't log in to the portal", it lands on login-bloqueado. No conversational search or LLM query rewriting needed.

    Redis queue and a single worker

    The GPU serves one generation at a time. The API returns 202 with a job_id immediately and the channel polls; the worker drains the queue serially, with graceful shutdown on SIGTERM. More throughput means more workers on more GPUs.

    Prompt-driven validation

    The system prompt states the programme's domain and scope explicitly, so the model does not import meaning from brand names, and questions outside that scope get a clear refusal instead of a guess.

    Knowledge in its own repository

    Skills change with operations, not with code. They live in seliga-rag-content, mounted as a volume; make ingest-skills reindexes and the next query already uses the new vector. Descriptions stay in Portuguese, because that's what gets embedded.

    One skill

    The frontmatter drives retrieval; the body becomes instructions for the model. The description carries the words participants actually type. Shown in Portuguese, as in production.

    Fig. 02skills/antecipar-boleto/SKILL.md
    ---
name: antecipar-boleto
description: Use quando o participante quer
  antecipar, adiantar ou mudar a data de
  pagamento de um boleto agendado.
---
# Antecipar boleto

## Quando usar
- "quero pagar antes", "adiantar o boleto"

## Procedimento
1. Pedido até 3 dias úteis antes
   do vencimento, antes das 12h.
...

    The API

    Asynchronous, authenticated with X-API-Key. The channel never blocks waiting for the GPU.

    Fig. 03REST · FastAPI
    POST /chat
{ "session_id": "c1f4…", "message": "esqueci a senha" }
202 { "job_id": "550e…", "status": "queued" }

GET /chat/status/550e…
200 {
  "status": "done",
  "answer": "Sem problema! Na tela…",
  "sources": [
    { "type": "skill",
      "name": "login-bloqueado",
      "score": 0.91 }
  ]
}

DELETE /chat/{session_id}   204
GET /health                  200

    Stack

    LayerChoiceWhy
    EmbeddingsBAAI/bge-m3Multilingual and strong in Portuguese, 1024 dimensions, runs locally via sentence-transformers.
    RerankerBAAI/bge-reranker-v2-m3Cross-encoder from the same family; scores the question–passage pair, not just distance.
    VectorsPostgres 16 + pgvector, HNSWOne database for vectors, sessions and metadata. No extra service to operate.
    LLMQwen2.5 14B · OllamaFollows instructions far better than 4B models; OpenAI-compatible API, so swapping models is one variable.
    QueueRedis 7Serialises GPU access; jobs expire after one hour.
    APIFastAPI + UvicornSmall contract, OpenAPI docs at /docs.
    Ingestiondocling, markitdownPDF, DOCX, PPTX and XLSX become markdown, cached by mtime, before chunking.
    IntegrationMCP server (FastMCP)The same index, queryable from Claude, Cursor and other MCP clients.
    DeployDocker ComposeSame compose on a Mac (LM Studio) and on the server; the GPU comes in through an override file.

    In production: Se Liga na SKY

    An incentive and points programme for employees. Questions arrive on WhatsApp and go straight to the API: portal access, missing points, redemption options and deadlines, prepaid cards, bill payments.

    Skills were written from real support conversations, using the human agent's tone as the reference. The operations team reads reports built on the sources trail: which skills fire most, which never fire, where refusals run high.

    The first thing the reports showed: access is the topic of people locked out. Blocked logins and forgotten passwords come precisely from those who can't get in, and that became a new first-access skill.

    What laloop delivers

    Fixed-scope projects or engagement by scope. From prototype to a monitored system in production, with code and documentation handed over to your team.

    RAG assistants
    Customer support, internal knowledge bases, operations support. Retrieval with reranking, thresholds calibrated on real data, and refusal when there is no source.
    Self-hosted LLMs
    Model and quantisation chosen for your hardware, Ollama or vLLM on GPU, OpenAI-compatible API. For when data cannot leave the company.
    Evaluation and observability
    Source and score trail per answer, coverage reports, test sets to calibrate thresholds before you change the model or the prompt.
    Infrastructure and SRE for AI
    Docker, Kubernetes, Terraform, GCP and AWS. Queues, workers, zero-downtime deploys and runbooks your team can operate.
    Agent integration via MCP
    Expose internal knowledge and systems as tools for Claude, Cursor and other clients, with the same index and the same permissions.

    Have an AI project stuck at the prototype stage?

    Tell us the problem, the data and where it needs to run. We'll come back with an architecture and scope proposal.

    Email contato@laloop.ai