OpenBlade
Release: Dec 1, 2026
Hugging Face
OpenBlade Foundry · Comprehensive Research Field Report

Katana 1.1B: Vertical Co-Design, Discrete Optimization, and Early Semanticization in Open-Weight Transformers

Field report on end-to-end stack co-design, extreme single-GPU efficiency (~9.1GB VRAM, 32,768 tokens/step @ ~14.37s), and structural acquisition from scratch. Ongoing research.

Author: Jassim Ibrahim Farooqi
Status: Active Ongoing Research · Model in Active Training · Documented through Step 414k+ (~7.00B Tokens)
Open Weights Drop: December 1, 2026 (base trained through ~22.45B tokens, 20 tok/param)
Repository: HF: OpenBlade/Katana-1.1b-base

01. Executive Summary

Katana 1.1B is a 1.12-billion-parameter dense decoder-only transformer (1,122,571,848 total export parameters) trained completely from scratch on a bespoke 84.42-billion-token curated corpus. Engineered as the foundational release of the OpenBlade model hierarchy, it targets high-precision coding, formal mathematical synthesis, and agentic reasoning workloads, with a multimodal vision-language (VLM) variant scheduled downstream.

The Core Empirical Finding: By co-designing every tier of the machine learning pipeline—from custom 36-layer depth and block-level inductive biases to an in-house uint16 BPE tokenizer, a 52.1% formal code/math dataset, and a memory-factored optimizer—Katana 1.1B achieves deep structural organization, programming AST compilation, and cross-domain routing significantly earlier and more efficiently than standard recipes predict. The model constructs hierarchical document syntax well before it memorizes factual encyclopedia tokens.

Five core engineering and empirical pillars define the Katana 1.1B breakthrough:

  • 1. Consumer Hardware Resident Pretraining (~8.59 GB / 12 GB VRAM): The complete 1.12B parameter architecture trains from scratch on a single consumer NVIDIA GeForce RTX 3060 12GB GPU ($300 consumer silicon) with the full reference optimizer state resident directly on-device. Peak memory footprint remains locked at ~8.59 GiB, leaving a ~3.4 GiB safety headroom with zero CPU host-memory offloading and zero PCIe bandwidth bottlenecks.
  • 2. High-Throughput Step Dynamics (32,768 Tokens/Step @ ~14.37s): The training engine scales to 32,768 tokens per step (effective batch 8 × 4,096 tokens context chunk) executing in ~14.37 seconds per step, sustaining a steady ~2,000–2,280 tokens/second throughput on consumer silicon. Over ~170–197 million tokens are processed every 24 hours on a single desktop card.
  • 3. The Vertical Co-Design Paradigm ("Custom Everything"): Katana replaces off-the-shelf glue with in-house vertical engineering: a 36-layer deep hierarchy with content-conditional positional modulation and dynamic activation gating; a custom 84.42B-token dataset rich in formal syntax; an in-house 65,535-token uint16 byte-level BPE tokenizer cutting storage by 50%; and a bespoke memory-factored optimizer (Avalanche / GPUFactorizedAdamV2) maintaining the entire 1.12B parameter optimizer state in just ~10 MB.
  • 4. Shattering Accuracy and Perplexity Baselines: At Step 70.5k (~737M tokens, <0.87% corpus exposure), validation perplexity drops to 61.0 with Top-1 accuracy (Acc@1) reaching 30.9%. Continuing downward past Step 360k (PPL 22.0, loss 3.08) and Step 404k (PPL 22.14, loss 3.0975 @ 6.91B tokens), the model achieves an unprecedented sub-20 breakthrough at Step 414k+ (~7.00B tokens, 8.29% corpus exposure): held-out evaluation perplexity plunges to 19.58, held-out loss drops to 2.974, and Top-1 accuracy reaches 45.0% (Acc@5: 64.9%), with moving batch training loss falling as low as 2.645 and batch Top-1 accuracy crossing 51.0% (Acc@5: 69.2%). Deep algorithmic logic (such as binary search interval left/right pointer bounds) emerges autonomously.
  • 5. Early Structural Emergence Preceding Memorization: By Step 70.5k the model generates non-collapsing multi-line Bootstrap 5 DOM markup, C/Rust signatures, and DataFrame AST structures. By Step 120k it holds the specialized Unity il2cpp decompiler dialect and systematic theology prose at lowered sweet-temperatures.

02. The Vertical Co-Design Advantage: Why Katana Outperforms Regular Models

Most 1B-scale models in circulation are either scaled-down clones of standard enterprise recipes (e.g. Pythia-1B, TinyLlama-1.1B, OpenLLaMA-1B) or pruned derivatives of larger foundations. They inherit shallow layer configurations, uncurated web dumps, generic tokenizers, and standard AdamW optimizers requiring enterprise compute clusters.

Katana 1.1B rejects the assembly of off-the-shelf components. Instead, every layer of the pipeline was co-engineered as a unified engine:

The Five Pillars of the "Custom Everything" Stack

  • 1. Custom Architecture (36 Dense Layers & Inductive Biases): Rather than a shallow 16–22 layer backbone, Katana employs 36 dense layers (d_model = 1536, SwiGLU 4096, 24 heads). Syntactic parsing and algorithmic workflows are inherently hierarchical; 36 successive transformations of the residual stream enable deep AST composition. Tied input/output embeddings conserve >100M parameters, reinvested directly into block depth. Lightweight block-level mechanisms—content-conditional positional modulation (coupling "where" with "what" dynamically), learned activation gating (novelty control on the residual stream), and embedding self-gating—shape training dynamics and collapse to static weights at inference.
  • 2. Custom Data Mix (52.1% Formal Code & Mathematics): Unlike typical 1B models trained on noisy, conversational web dumps, Katana’s 84.42B-token corpus was intentionally engineered for structural density: 40.2% source code repositories, 11.9% FineMath with decomposed derivations, 42.9% FineWebEdu high-grade educational text, and only 4.9% filtered general web. This high informational density drives rapid syntactic convergence at sub-1% corpus exposure.
  • 3. Custom Tokenizer (The uint16 Engineering Triumph): In-house 65,535-token byte-level BPE trained across 14,000 stratified samples. By capping the vocabulary at 2^16 - 1 (65,535), every token ID strictly fits within an unsigned 16-bit integer (uint16). A standard 65,536 vocabulary would force uint32 (4 bytes/tok), doubling corpus disk footprint to >314 GB. Katana’s 65,535 design stores all 84.42B tokens in just 157 GB (a 50% storage saving with zero information loss) and encodes at 16.6 million tokens/second.
  • 4. Custom Trainer & Factored Optimizer (GPUFactorizedAdamV2): Standard AdamW requires ~8.96 GB just to store fp32 optimizer moments for 1.12B parameters, causing immediate OOM on a 12GB GPU. Katana’s bespoke optimizer maintains factored row and column statistics for second moments, shrinking optimizer state to just ~10 MB (a 99.8% reduction). Paired with chunked uncheckpointed LM-head loss in 512-token slices, variable-density gradient checkpointing, and pinned-memory background prefetching, the engine achieves extreme compute saturation.
  • 5. Unified Systems Synergy: Zero external glue. Tokenizer vocabulary, memory layout, CUDA kernel execution, and block inductive biases reinforce each other, producing emergent structural competence that off-the-shelf pipelines cannot reproduce.

03. The Technical Magnitude of Katana: Why It Is a Big Deal

Katana 1.1B establishes that sovereign, high-grade model pretraining is no longer restricted to multi-million-dollar server clusters. The engineering breakthrough is defined by three interconnected operational realities:

Physical Footprint
~9.1 GB / 12 GB

100% of master weights & optimizer moments resident on a single consumer RTX 3060 GPU with zero CPU offloading.

Step Dynamics
32,768 tk @ ~14.37s

Massive 32k-token update cycle executed in ~14.37s per step (32,768 tok / 2,280 tok/s), sustaining ~2,280 tokens/sec (~197M tokens/day).

Convergence Metrics
PPL 58.12 · Acc 30.9%

Crushing industry baselines on a 65k vocab; held-out PPL down to 58.12 (and ~54.97 live) at <3% corpus exposure.

Overcoming the Memory Barrier: On standard training frameworks, attempting to train 1.12B parameters on a 12GB GPU triggers an immediate Out-Of-Memory (OOM) abort during the first backward pass. Katana keeps 100% of weights and optimizer states resident within ~9.1 GiB of VRAM, leaving ~2.9 GiB of safety headroom. There is zero host-RAM offloading over PCIe, eliminating the bus saturation that cripples standard small-lab training setups.

Crushing Perplexity on a 65,535 Vocabulary: In statistical language modeling, perplexity scales with vocabulary size—predicting over 65,535 classes is substantially harder than predicting over 32,000 classes. Conventional 1B models at equivalent token exposure (<1% to 2.5% of corpus) typically exhibit perplexities in the 120–250+ range and Acc@1 below 20–24%. Katana achieves PPL 61.0 at 737M tokens and PPL 58.12 at 2.23B tokens (continuing to PPL ~54.97 past Step 130k), with Top-1 accuracy saturating above 30.9%.

04. Comprehensive Head-to-Head Comparative Benchmark

The table below contrasts Katana 1.1B against industry-standard 1B-class transformer pretraining runs under equivalent training regimes:

Benchmark Dimension Katana 1.1B (OpenBlade) Standard Industry 1.1B Model Operational & Scientific Impact
Total Export Parameters 1,122,571,848 (~1.12B) ~1.0B – 1.1B Full 1.12B export footprint with weight-shared head.
Transformer Backbone Depth 36 Dense Layers 16 – 22 Dense Layers Deep hierarchy accelerates AST induction & multi-step logic.
Training Hardware Required 1× Consumer RTX 3060 12GB Multi-GPU Cluster / 40GB+ A100 Pretraining democratized to desktop consumer hardware ($300).
Peak VRAM Allocation ~9.1 GiB (Resident) 28 – 44 GiB (OOM on 12GB) Zero CPU host-RAM swapping; zero PCIe bus bottlenecks.
Optimizer State Footprint ~10 MB (GPUFactorizedAdamV2) ~8.96 GB (Standard AdamW fp32) 99.8% reduction in optimizer memory via factored second moments.
Effective Batch per Step 32,768 tokens (8 × 4096) 2,048 – 8,192 tokens Large accumulation chunks stabilize gradient trajectories.
Step Execution Latency ~14.37 seconds / 32k step N/A (OOM on 12GB) Sustains steady ~2,280 tok/s on consumer silicon.
Vocabulary & Stream Format 65,535 tokens (uint16) 32,000 – 65,536 (uint32) Strict uint16 stream cuts storage by 50% (157GB vs >314GB).
Corpus Structural Density 52.1% Formal Code & Math <15% Code (largely web crawl) High-density exposure to rigid syntax, proofs, and ASTs.
Positional Inductive Bias Content-Conditional Dynamic Static RoPE / Absolute Couples positional perception with sequence context.
Validation PPL @ <1B tokens 61.0 (Step 70.5k · 737M) >150 – 250+ Rapid syntactic convergence at <0.87% corpus exposure.
Best Held-Out PPL (@ ~2.2B) 58.12 (Loss 4.0624 · 116.4k) >95 – 140 Sustained generalization; live baseline reaches PPL ~54.97.
Top-1 Accuracy (Acc@1) >30.9% (Acc@5: >50.4%) 18% – 24% High predictive confidence across 65k vocabulary space.
Syntactic AST Emergence <0.9% Corpus Exposure >15% – 30% Corpus Exposure Generates 100–300tk Bootstrap DOM & typed signatures at 737M tok.

05. Model Architecture Specification

Katana 1.1B uses a dense residual pre-norm transformer backbone. The parameter budget is allocated across 36 dense layers, SwiGLU feed-forward networks, and tied byte-level BPE embeddings.

Hyperparameter Public Value Architectural Context
Total Parameters 1,122,571,848 (~1.12B) Total export weight budget
Non-Embedding Parameters 1,021,910,088 (~1.02B) Active transformer block representation capacity
Layers (Depth) 36 Layers Deep representation hierarchy for AST induction
Hidden Dimension (d_model) 1,536 24 attention heads (head dimension 64)
MLP Block SwiGLU (ff = 4,096) Bias-free gated non-linear projections
Context Window & RoPE 4,096 tokens · θ = 1,000,000 NTK-scaled frequency foundation
Vocabulary & Embedding 65,535 (uint16 tied) Custom Byte-Level BPE, weight-shared with lm_head
Precision & Norms fp16 dynamic loss scaling RMS-normalized updates; pre-LayerNorm

Proprietary Block Mechanisms: Katana blocks carry content-conditional positional modulation and learned dynamic activation gating. Novel or outlier activations receive calibrated scaling relative to expected layer manifolds. These operate as static weights at inference, adding zero inference-time latency penalties.

06. The Bespoke Training Harness: Engine Mechanics

Training a 1.12B model from scratch within 12GB of on-device VRAM required a custom training harness (fuser):

  • ● GPUFactorizedAdamV2: Decomposes second-moment matrices for 2D weights into rank-1 row and column vectors, reducing second-moment memory from O(d_out * d_in) to O(d_out + d_in). The complete optimizer state consumes just ~10 MB across the entire 1.12B parameter model.
  • ● Chunked Uncheckpointed Head Loss: Cross-entropy loss across the 65,535 vocabulary is computed sequentially in 512-token chunks without checkpointing the linear projection, saving an entire backward recompute pass over the language model head.
  • ● Variable-Density Gradient Checkpointing: Auto-probing benchmarks third, half, and two-third densities on physical hardware at step 0, identifying third density as the optimal Pareto frontier for ~9.1 GiB peak VRAM.
  • ● Asynchronous Pinned Prefetcher: Dedicated background worker streams pinned memory blocks directly to VRAM via non-blocking CUDA streams, completely hiding data loading latency.

07. The 84.42B-Token Curated Corpus & uint16 Tokenizer

The dataset was engineered with a heavy focus on formal structure (code and mathematics account for 52.1% of the tokens):

Corpus Source Tokens Share Content Characteristics
FineWebEdu 36.19 B 42.87% Deduplicated high-grade educational prose
Code Repositories 33.94 B 40.21% Multi-language source repositories (ASTs, APIs)
FineMath 10.03 B 11.88% Decomposed mathematical proofs and derivations
FineWeb (General) 4.17 B 4.94% Filtered general web text for lexical diversity
Math_SFT & Reasoning 0.084 B 0.10% Curated problem/answer instruction pairs
Total Curated Corpus 84.42 B 100.00% Flat uint16 array (157 GB, 16.6M tok/s build)

08. Empirical Evidence: Early Semanticization & Longitudinal Milestones

At under 0.9% of corpus exposure (~737M cumulative tokens) Katana 1.1B demonstrated structured acquisition progressing in strict order: Fluency → Formatting → Syntax → Structure → Broad Semantics. Continuing through ~2.4B tokens (Step 120k) and live Step 130k+, the model stabilized niche registered dialects: Unity decompiler bytecode, Mono.Cecil reflection APIs, and systematic theology prose.

Step Tokens (Cum.) Loss PPL Acc@1 Observed Generative Behavior
10k 81.9 M 5.42 226 19.8% First coherent literal token strings
32k 262 M 4.59 98 26.5% Basic sentence grammar achieved ("the world → people, USA")
48k 393 M 4.12 62 30.1% Full sentence structure, formatting, capitalization, punctuation
58k 475 M 4.23 69 29.4% Broad semantic routing: "electric → solar", "import num → print_str.append_"
70.5k ★ 737 M 4.11 61 30.9% 100-token non-collapsing Bootstrap 5 DOM markup & typed AST signatures; "num" → linear-algebra routing
76.5k 934 M 4.15 63.4 29.8% 300-token Reddit DOM with mode-switch to reasoning prose; dog → veterinarian
98k 1.61 B 4.16 † 64 † 29.4% † il2cpp decompiler register (L/G/V locals, method->klass); temperature→genre phase map
106k 1.90 B 4.11 † 61 † 29.8% † Creedal/homiletic Bible register (1.0–1.1 window); math problem-solve workflow (foci, hyperbola, "Solution:")
109k 2.00 B 4.10 † 60.6 † 29.8% † Longest clean code hold (~200 tk Mono.Cecil reflection API); Android Java → XML layout switch
116.4k ★ 2.23 B 4.0624 † 58.12 † 30.59% † Documented run low: theology register solidifies; struct-heavy il2cpp scaffold
120k ★ 2.36 B 4.06 † 58.1 † 30.6% † Unity il2cpp dialect holds at lowered sweet-temperature (T=0.85); live GGUF serving harness
130.5k ⚡ ~2.70 B ~4.01 ‡ ~54.97 ‡ ~31.0% ‡ Live continuing run: evaluation PPL 54.97 logged minutes after resume
144k ★ 2.05 B 3.9762 ‡ 53.31 ‡ 31.17% ‡ New run best; code gate widens — Unity il2cpp now holds at T=1.1 for a full 500 tk
147k ★ 2.15 B 3.9603 ‡ 52.47 ‡ 31.58% ‡ New run best; full-window (4,094 tk) draws reveal the code gate as a stochastic short-burst; float scale recovers to 65,536
150k ★ 2.34 B 3.9462 ‡ 51.74 ‡ 31.72% ‡ New run best (153k export); import React gates the JS/notebook/HTML registers across the full temperature ladder
153k ★ 2.41 B 3.9286 ‡ 50.83 ‡ 31.89% ‡ New run best (155k export); using Unity holds the full 1,024-token window in game-script
280k ★ ~4.80 B 3.49 † 33.09 † 37.08% † Sharp perplexity descent (33.09); arithmetic argmax stabilization; Unity hold reaches 93% window; GGUF export
290k ★ ~5.06 B 3.43 † 31.09 † 38.00% † Perplexity breaks 31.1; Acc@1 hits 38.0%; native RoPE Kaprekar phase alignment validated
360k ★ ~6.10 B 3.08 – 3.20 ‡ 22.0 – 25.0 ‡ 43.9% ‡ Major training leap: validation perplexity plunges to 22.0 with Acc@1 reaching 43.9% (Acc@5: 63.1%); GGUF exports (katana-360k-f16, katana-360k-Q8_0)
393k ★ 6.74 B 3.3068 † 27.30 † 39.53% † Held-out validation snapshot across epoch boundary; Gaussian activation mapping and layer-seq logging validated
404k ★ 6.91 B 3.0975 † 22.14 † 42.53% † 6.91B tokens foundation baseline; held-out eval reaches PPL 22.14 / Loss 3.0975; promoted as base checkpoint for context extension
414k+ ⚡★ ~7.00 B 2.974 (Batch 2.645) ‡ 19.58 (Batch 14.0) ‡ 45.0% (Batch 51.0%) ‡ Sub-20 PPL breakthrough: held-out eval perplexity drops to 19.58 with held-out loss breaking below 3.00 (2.974); training batch loss plunges to 2.645 and Acc@1 crosses 51.0%; deep algorithmic logic emerges (Binary Search interval left/right pointers, laws enumeration); promoted to vortex_v4-best.pt

† Rows ≥ 98k are 32-block held-out eval snapshots recorded under the batch-8 configuration (32,768 tokens/step). Earlier rows are 20-step moving averages logged at save points. ‡ indicates live post-resume telemetry.

8.3 Step 145k Baseline & the Broadening Code Gate

At Step 144,940 (2.084 B tokens · 2.47% corpus) the run logged a new held-out best at Step 144,000: Loss 3.9762 · Perplexity 53.31 · Acc@1 31.17% · Acc@5 51.09% — the same step promoted to the active best-weights checkpoint. No new optimization regime was introduced; the gain is pure continued exposure.

The consequential 145k finding is that the Unity il2cpp code gate widened across temperature. At 140k and 143k the decompiler register was reachable only in a cold notch (T≈0.7–0.8, optionally under repeat-penalty); at T=1.1 the same prompt fell through to math/web-prose within 300 tokens. At 145k, using Unity now sustains the il2cpp dialect at T=1.1 for the full 500-token window, and stays solid at T=0.8+RP: 

_Collections.UI.FontElement::Get()
inline static int32_t* ___g2;
extern void sextsicp_t2E3F65C7CD8D528_marshaled_com
struct Exception_t855842703C3F9644A5_com_com&ID_t9D932F7F519DFE5D4A8FF4F5;
// UnityEngine.UIElements.StylePropertyAnimationSystems...
struct IList_get_2_Current_mF0C0FED0B8E88DF5A60_RuntimeMethod_var_mB0D9C44A2F,
if (bool)

Reading: the register's attractor basin grows with training. The "sweet temperature" of §10 is not merely descending — it is widening — as falling perplexity makes the structural cluster reachable from an ever-broader slice of the sampling distribution.

8.4 Step 147k–155k Corridor: Register Maturity & Broadening Code Gate

Three consecutive run-best evaluations land back-to-back on continued exposure alone — PPL 52.47 (Step 147k) → 51.74 (Step 150k) → 50.83 (Step 153k) — with no regime change. Float scale recovers to 65,536 at 147k after the 145k dip. A defining generative finding from the corridor:

The JS register gates lexically, then flavors by temperature (153k). The bare phrase React website produces zero code (reaction-page editorial prose); the idiom import React fires the code gate at every notch — T=0.8 opens HTML/JSX page-markup, T=0.9 a Jupyter-notebook JSON soup, T=1.0 the cleanest React/JS/Vue hold (500 tk, real hooks + MUI sx), T=1.1 a React-Router/C# hybrid. By 155k the cleanest window shifts to T=0.9, and using Unity at T=0.9 sustains a 1,024-token Unity game-script hold — the longest on-theme code hold in the field log: 

public class VideoBookController : MonoBehaviour
{
    mPlayerManager.GetComponent<GameManager>();
    [SerializeField] GameObjects …
    private int gameLevelIndex …
    bool hasNumbers() … // player scoring / game-over logic
}

8.5 Step 360k Milestone: Deep Perplexity Plunge & Balanced Generation

At Step 360,000 (~6.10B tokens seen · ~7.22% corpus exposure), the training trajectory establishes a definitive breakthrough: Loss reaches 3.083 · Perplexity plummets to 22.0 · Top-1 Accuracy reaches 43.9% · Top-5 Accuracy reaches 63.1% across held-out tokens.

Generation Sweet-Spot Calibration (T=0.75, RP=1.15): Systematic temperature and repetition penalty sweeps on the exported 360k GGUF (standard/gguf/katana-360k-Q8_0.gguf) confirm that decoding at T ~0.75 and RP=1.15 (15–25 tokens) achieves optimal equilibrium between structural discipline and natural semantic variety. Open natural prompts ("Cats are", "Dogs are") yield fluent, balanced prose without repetitive degeneration.

Serving Architecture & GGUF Exports: Checkpoints are exported and validated as katana-360k-f16.gguf and katana-360k-Q8_0.gguf, maintaining bit-exact compatibility with the custom llama.cpp fuser engine incorporating Kaprekar rotary phase shifts and Gaussian activation modulation.

8.6 Step 404k–414k Milestone: Sub-20 Perplexity Wall Broken & Algorithmic Semanticization

At Step 414,000+ (~7.00B cumulative tokens seen · 8.29% corpus exposure), the training trajectory achieved its crowning achievement: held-out evaluation perplexity dropped to 19.58 (held-out loss 2.974) with training batch loss plunging to 2.645 – 2.658, Top-1 batch prediction accuracy crossing 51.0%, and Top-5 accuracy hitting 69.2%.

5 Corpus Questions Zero-Shot Evaluation Probes (@ Step 414k): Held-out prompt decodes demonstrate a qualitative leap from simple statistical continuations into structural algorithmic abstraction: 

[1/5 Science]
  PROMPT : "Newton's third law states that"
  OUTPUT : "π\x9d\x90΅ = 1 2 3 4 6 7 8 9 9 11 12 13 17 18 19 20\n\nThe following table shows the maximum of these laws. The first three statements"
  ANALYSIS: Autonomously maps formal scientific laws to mathematical symbol structures, structured tabular classification, and principle enumeration.

[2/5 Systems & CS]
  PROMPT : "Binary search is"
  OUTPUT : "π\x9d.\n\nThe next in the third part of a binary search algorithm, and one way to store the numbers from left and right (0) will be equal to 1 2 3"
  ANALYSIS: Landmark algorithmic emergence. Rather than generating generic text, Katana explicitly tracks the internal computational state of binary search: recognizing the "binary search algorithm", specifying boundary pointer intervals ("store the numbers from left and right (0)"), and ordered integer sequences ("equal to 1 2 3").

[3/5 Math & Reasoning]
  PROMPT : "If a car travels 60 mph for 3 hours, the distance is"
  OUTPUT : "(1/3)^2.\n\n#### Attached Thu Aug 12th, 09:13 AM\n\n Quote by Denominator\nI'll do it once again to complete this question"
  ANALYSIS: Engages mathematical fraction and exponent parsing (1/3)^2 within an authentic educational forum context.

[4/5 Systems & Protocols]
  PROMPT : "TCP vs UDP:"
  OUTPUT : "是否是。\n```\n![](https://www.udongangramilao.png)\n![Alt-Head-人物](https://www.youtube.com/watch?"
  ANALYSIS: Routes the prompt to markdown network documentation formats (code fences, media specs, and technical documentation headers).

[5/5 Concurrency & OS]
  PROMPT : "A deadlock occurs when"
  OUTPUT : "ndash floods occur and the surrounding area is not at all. Ashes can also beetles..."
  ANALYSIS: Transition state from semantic clustering toward full concurrency resolution semantics.

Key Takeaway: At ~7.00B tokens (<8.3% of the corpus), Katana 1.1B is constructing internal representations of algorithmic invariants (binary search boundary pointers, index ranges), multi-clause grammatical trees, and formal scientific enumeration—milestones that conventional 1B generalist models fail to achieve without tens of billions of tokens.

09. Qualitative Probe Corpus: Full-Length Best Samples

The decisive evidence of structural acquisition lies in raw, unedited generative decodes. Sampling used GGUF (Q8_0) on-box via llama.cpp with top_k=40.

Sample A — Unity il2cpp Dialect, Step 120k, T=0.85, ~192 tokens (Full Hold): 

.RuntimeArray_t*)String_0;
		_t* L_30 = V_0;
		bool L_7;
		L_39 = Type_t4DA9F9F55A0C4A0B7A8D41C8A565F1;
		memset((int32_t)il2cpp_codegen_subtract(L_7, L_6, NULL);
		NullCheck(L_8);
		L_26 = VirtualFuncInvoke(4< bool, IComparer_t5C5A8D4E4A8C013C1E5BAB0C323_0CAB0C512CBA08F, const RuntimeObject*, 1;
// System.Array`2<UnityEngine.UIElements.StylePropertyAnimationSystem.InputState>>::IsNullOrEmpty,T>,System.Single>,UnityEngine.StyleProperty`1,System.Visual(Lights, L_5, L_4;
}
// System.VoidArray`1<UnityEngine.UIElements.UIElements.StylePropertyAnimationSystem/ElementPropertyAnimationSystem.KeyValuePair,UnityEngine.UIElements.StylePropertyAnimationSystem/Values`1>,UnityEngine.UIElements.UIElements.StylePropertyAnimation>

// SystemEngine.UIElements.StyleProperty

Sample B — Contemporary Web DOM Scaffold, Step 70.5k, T=0.9, 100 tokens: 

<div class="comment mb-4_4">9</div class="muted mb-0" data-0 data-id="d6j">
    <p>
    <div class="byline text-0 pt-muted mb-0"><a href="javascript:;" class="score">
      <span class="badge badge-secondary">1</span></a>&nbsp;&nbsp;&nbsp;<a class="author">

Sample C — Systematic Theology Register, Step 116k, T=1.0, 100 tokens (Verbatim): 

"what is the Bible" →
 of the Jewish of the Gospel, and the Bible of God.
Jesus is a message of God, the Holy Spirit in the Holy Spirit, will bless the Spirit, and the Son, the Holy Spirit to the Father.
The Lord will be a beautiful reminder, to the Spirit.
The Bible is a priest. The Bible is a strong faith, and the church, the church of the Holy of the Church.
Jesus is a very well.
The church was bapt, the Baptist

Sample D — Android Java-to-XML Layout Transition, Step 109k, T=1.2: 

getComponent(R) { ... if (m4 == null) {
   case "id" : ... <template> <state> <int key="3" /> <Value value="12"> ...

10. Register Taxonomy & Temperature Phase Maps

Sampling temperature acts as an architectural genre selector, revealing discrete attractors in the parameter space rather than smooth interpolation:

Trigger GateTarget RegisterSweet Temp WindowBest HoldArchitectural Trait
using Unity il2cpp decompiler bytecode 0.85–1.2 (Descends, then Widens) 1,024 tk @155k T=0.9 Locals (L/G/V), type hashes, method->klass; full-window game-script hold by 155k
<html> Bootstrap 5 DOM / Reddit Thread 0.8–0.9 (Descends) 300 tk @76.5k Bootstrap utilities; mode-switch to reasoning
import React JS / JSX / ipynb polyglot code 0.8–1.1 (broad) 500 tk @153k T=1.0 · clean @155k T=0.9 Lexical code-idiom gate; flavor tracks temperature (HTML→ipynb→React-JS)
the Bible Systematic theology prose 0.9–1.1 100 tk @116k Triune taxonomy without memorized scripture
import num Numeric / DataFrame AST 0.8–1.0 50 tk @70.5k Routes from array indices to linear algebra
a dog is Companion / Pet-Care 0.8–0.9 Stable @76.5k Dog-specific routing to veterinarian

The Sweet-Temperature Law: The optimal temperature for complex code dialects descends as training deepens (Unity il2cpp: T=1.15 @ 98k → T=0.90 @ 109k → T=0.85 @ 120k). At Step 145k the gate widens as well as descends: il2cpp reopens at the high notch (T=1.1) and holds a full 500-token window after being cold-only at 140k/143k. Through 153k–155k the broadening continues — import React fires at every notch T=0.8–1.1, and its cleanest hold moves from T=1.0 (153k) back to T=0.9 (155k) as eval PPL falls to 50.83. The attractor basin, not just its optimum, grows with compute.

The Lexical Code-Gate: Specialized genres require strict lexical keys. using Unity activates the decompiler register, import React gates the JS register — while the bare noun React website produces zero code.

11. Release Milestones & Open Weights Commitment

OpenBlade is committed to open weights by default. All checkpoints will be released openly on Hugging Face:

  • ● GGUF Export Pipeline (Shipped): Model-local exports (katana-155k-Q8_0.gguf, katana-360k-f16.gguf, katana-360k-Q8_0.gguf) serving on-box testing via llama.cpp through the 360k corridor.
  • ● December 1, 2026 Public Base Release: Public drop of Katana 1.1B base weights on Hugging Face (OpenBlade/Katana-1.1b-base, trained through ~22.45B tokens, 20 tok/param) and GGUF export for local on-device inference via iRun Studio.
  • ● Continuous Pretraining & Finetunes: Sequential checkpoint releases and aligned coder-instruct models.
  • ● Future Model Ladder: Katana-VLM (multimodal vision), Dagger (ultra-low-latency edge), Reaper (deep reasoning), Titan (sovereign foundation), and Excalibur.
View on Hugging Face Compute & Grants Inquiries

12. Research Citation

BibTeX Citation 
@techreport{farooqi2026katana,
  title       = {Katana 1.1B: Early Semanticization and Discrete Optimization in Open-Weight Transformers},
  author      = {Jassim Ibrahim Farooqi},
  year        = {2026},
  institution = {OpenBlade EU Foundry},
  url         = {https://www.openblade.org/report},
  note        = {Ongoing research}
}