Home / Project guide

Project guide

Project Guide | Tool-Call Fine-Tune Lab

Free tool-call dataset boundary checklist for fine-tune preparation and eval reviews. This guide organizes the repository's original implementation notes for LLM evaluation engineers and dataset reviewers.

Reviewed 2026-07-28. This page is derived from checked-in repository evidence and links back to its source.

Agent Reliability Audit Surface

QLoRA fine-tuning of Qwen2.5-7B-Instruct for tool-calling, evaluated on BFCL, served with vLLM.

System Overview

A tool-call adaptation lab that connects training, benchmark evaluation, and serving notes into one model-readiness surface.

AreaDetails
UsersAI platform teams, model-evaluation teams, applied ML engineers, and developer-tool builders.
Technical pathStart with the demo, then inspect the training configuration, evaluation pipeline, architecture notes, and quality gate.
System scopeOpen-weight training notes, benchmark evaluation, QLoRA/PyTorch framing, vLLM serving notes, and tool-calling datasets.
Operating boundaryFine-tuned models need dataset licensing checks, eval coverage, and deployment controls before production use.
Evaluation pathRun the available evaluation scripts and inspect the training config, benchmark reports, and serving notes.

Evaluation Path

Architecture Notes

Motivation

stage-pilot gets tool-calling success from 25% to 90% via middleware (parser recovery + retries). This repo tries to close the remaining gap by fine-tuning the model itself to produce better tool calls in the first place.

We use BFCL as the eval benchmark (not a homebrew test suite) and serve through an OpenAI-compatible endpoint that plugs into stage-pilot with zero code changes.

Training

ItemDetail
Base modelQwen/Qwen2.5-7B-Instruct
MethodQLoRA (4-bit NF4 + fp16 LoRA adapters)
LoRA configrank=16, alpha=32, dropout=0.05, targets: q/k/v/o_proj
Data29,647 examples (BFCL v4 + Glaive-function-calling-v2), 80/10/10 split
HardwareKaggle T4 16GB (QLoRA), also works on A100 80GB
Training1 epoch, lr=2e-4, batch=1 x grad_accum=8, gradient checkpointing
TrackingW&B (optional)

Data Pipeline

BFCL v4 (GitHub)          Glaive v2 (HuggingFace)
  2,501 examples              63,218 examples
        \                        /
         \                      /
          merge + dedup + split
                  |
        29,647 total examples
     train: 23,716 (80%)
     val:    2,962 (10%)
     test:   2,969 (10%)

Security/reproducibility defaults:

Architecture

stage-pilot copilot
        │
        ▼
   OpenAI-compatible
   tool-call API
        │
   ┌────┴────┐
   │  vLLM   │  ← AWQ INT4 quantized
   │ server  │
   └────┬────┘
        │
   Qwen2.5-7B-Instruct
   + LoRA adapters
   (merged or loaded via PEFT)

Quick Start


## install
git clone https://github.com/KIM3310/tool-call-finetune-lab
cd tool-call-finetune-lab
pip install -e ".[dev]"


## with GPU deps:
pip install -e ".[gpu,dev]"
make data                   # download + merge + split (no GPU)
make train                  # QLoRA fine-tuning
make merge                  # merge LoRA into base
make eval                   # BFCL eval + comparison table
make quantize               # AWQ INT4
make serve                  # vLLM server
make smoke-test             # hit the server
make pipeline               # data -> train -> merge -> eval -> quantize

Docker

docker compose up vllm-server
docker compose run smoke-test

Default compose publishes the host port on 127.0.0.1 only. The container-internal vLLM process may still bind 0.0.0.0 so the smoke-test service can reach it on the Docker network.

Public host publishing uses the standalone production Compose file and requires an API key with at least 16 characters:

export VLLM_API_KEY="$(openssl rand -hex 32)"
VLLM_PRODUCTION_HOST_BIND=0.0.0.0 docker compose \
  -f docker-compose.production.yml up vllm-server

The API key protects the OpenAI-compatible routes. Keep the service behind a reverse proxy/TLS boundary and firewall rules that restrict endpoint access.

Kaggle

Full pipeline notebook: kaggle.com/code/doeonkim00/tool-call-fine-tune-lab-qlora-pipeline

Dev

make check                  # lint + typecheck + test
make test                   # pytest
make test-cov               # with coverage
make lint                   # ruff
make format                 # auto-format
make typecheck              # mypy
make help                   # all targets

Artifacts

ArtifactLink
Kaggle kernelkaggle.com/code/doeonkim00/tool-call-fine-tune-lab-qlora-pipeline
LoRA adapterhuggingface.co/KIM3310/qwen2.5-7b-tool-calling-lora
AWQ quantizedhuggingface.co/KIM3310/qwen2.5-7b-tool-calling-awq
W&B runwandb.ai/KIM3310/tool-call-finetune-lab

Layout

src/tool_call_finetune_lab/
  config.py              # hyperparams (dataclasses)
  data/                  # BFCL + Glaive download, parse, merge, split
  train/                 # QLoRA trainer + adapter merge
  eval/                  # BFCL runner, stage-pilot bridge, comparison
  quantize/              # AWQ quantization + inference bench
  serve/                 # vLLM launcher + smoke test
notebooks/
  kaggle_full_pipeline.ipynb
scripts/                 # download base model, push to hub
tests/

Requirements

See Also

stage-pilot -- middleware-level tool-calling reliability (25% -> 90% via parser recovery and retries). This repo is the model-level complement: teach the model to produce well-formed tool calls so middleware has less work to do.

Cloud + AI Architecture

Enterprise Productization

System Architecture

Service Architecture

Search And Service Surface