Katana 1.1B
Code, Math & Autonomous Agentic Core

Conventional 32-bit Adam requires 16 bytes of optimizer state per parameter (over 17.9 GB for 1.12B params), forcing distributed ZeRO-3 offloading across data center nodes. OpenBlade's co-designed training engine leverages factored second moments and memory-pinned gradient buffers to compress total steady-state resident memory down to ~9.1 GB on a single desktop NVIDIA GeForce RTX 3060.

Exact mathematical throughput sustained across 130,500+ pretraining steps without performance degradation:

• ✓ Zero PCIe Bottlenecking: All tensors, states, and gradients remain 100% on-device.
• ✓ No CUDA OOM Crashes: 2.9 GB safety margin accommodates long sequence dynamic activations.
• ✓ Sustained 2,280 tok/s: Maximize MFU on ampere tensor cores via fused kernel operations.

36 dense layers with 1,536 hidden dimensions and 12 attention heads. Replaces generic attention layers with:

• Content-Conditional Positional Modulation: Dynamic attention scaling that binds tokens based on syntax roles (parentheses, scope braces, indentation levels).
• Learned Dynamic Activation Gating: Non-linear thresholding that eliminates syntactic noise before intermediate projections.
• Embedding Self-Gating: Selective feature filtering that prevents rare token variance from corrupting shared latent space.

Strictly filtered for mathematical depth, low-level systems engineering, and formal logical coherence:

• 52.1% Code & Logic: Linux kernel sources, Mono runtime, Unity ECS, IL2CPP bindings, formal math proofs, and systems engineering.
• Zero Synthetic Hallucinations: 0% distillation from closed frontier LLMs. Retains authentic human debugging, commit histories, and edge cases.
• Deep Semantic Deduplication: MinHash LSH and strict heuristic linting filtering out boilerplate and auto-generated noise.

A 65,536 vocabulary optimized specifically for programming syntax, abstract syntax trees (ASTs), and mathematical logic:

• Atomic Indentation Tokens: Multi-space blocks (2, 4, 8 spaces) encoded as single atomic tokens, eliminating token bloat in nested code.
• Preserved Dot-Syntax Identifiers: Canonical namespaces (System.Collections.Generic) represented without fragmentation.
• 30-40% Sequence Compression: 100 lines of code require 35% fewer tokens than standard Llama/Mistral tokenizers.

Built from bare metal in PyTorch and CUDA C++ to bypass framework overhead:

Why vertical co-design outperforms horizontal modularity:

When the tokenizer knows the architecture's positional bias, the architecture knows the exact distribution of code tokens, and the optimizer's learning rate schedules match the gradient dynamics of the custom gating activations, every single FLOP directly increases parameter intelligence rather than overcoming inter-layer mismatches.

Execute directly via terminal with hardware-accelerated CPU threads or CUDA/Metal offload:

# Run Q8_0 near-lossless quantization
llama-cli -m katana-130k-Q8_0.gguf   -p "using Unity.Entities; public struct MovementSystem : ISystem {"   -n 384   -t 8   --temp 0.7   --top-p 0.9

Embed Katana directly into Python agentic pipelines or local IDE plugins:

from llama_cpp import Llama

llm = Llama(
    model_path="./katana-130k-Q8_0.gguf",
    n_ctx=4096,
    n_threads=8,
    n_gpu_layers=-1 # Full offload to GPU / Metal
)

output = llm(
    "using Unity.Burst;
[BurstCompile]
public struct",
    max_tokens=256,
    stop=["


"]
)
print(output["choices"][0]["text"])

Run Katana natively on macOS Apple Silicon (M1/M2/M3/M4) with full Metal acceleration and a modern GUI.

Package Katana into your local Ollama runtime with a single command:

FROM ./katana-130k-Q8_0.gguf
PARAMETER temperature 0.7
PARAMETER stop "<|endoftext|>"

• Q4_K_M (4-bit) ~720 MB RAM
• Q8_0 (8-bit) ~1.18 GB RAM
• FP16 (Full Precision) ~2.25 GB VRAM