# Project Proposal: SD Card-Driven Transformer Accelerator for Edge NLP

## Track: A (Digital / Mixed-Signal)

## Team: Chipathon Team

### Team Members

| Name | Role |
|------|------|
| Xuhe | Architecture & RTL Design |
| Hank | Memory Controller & Verification |
| CC | Layout & DRC/LVS |
| Sam | Integration & Testbench |

---

## Overview

A compact transformer accelerator for **edge NLP inference** that reads tokenized data directly from an **SD card** interface and performs forward-pass computation using quantized weights stored in on-chip SRAM. The design targets simple language modeling and keyword classification tasks on the GF180 MCU process, with a **hardware SD card interface** for data loading and **SPI interface** for result output.

Unlike CPU/GPU-based transformer inference that loads all weights and data into DRAM, this design implements a **streaming architecture** where:
1. SD card data (tokenized sequences) is streamed in real-time via an SD/MMC controller
2. Weights are pre-loaded to SRAM before inference
3. Compute is pipelined to minimize on-chip buffer requirements
4. Results (class probabilities or next-token predictions) are output via SPI

This enables **low-cost edge devices** that can perform transformer inference using only an SD card for input data and a simple SPI host interface — no CPU needed.

---

## Architecture

### System Block Diagram

```
┌──────────────────────────────────────────────────────────────────┐
│                    Transformer Accelerator                       │
│                                                                  │
│  ┌─────────┐    ┌─────────────┐    ┌───────────────┐           │
│  │ SD Card │───►│ SD/MMC      │───►│ Token Buffer  │           │
│  │ Interface│   │ Controller  │    │ (FIFO)        │           │
│  └─────────┘    └─────────────┘    └───────┬───────┘           │
│                                             │                   │
│  ┌─────────┐    ┌─────────────┐    ┌───────▼───────┐           │
│  │ SPI     │◄───│ Output      │◄───│ Attention     │           │
│  │ Host    │    │ Serializer  │    │ Engine        │           │
│  │ Interface│   │             │    │ (Quantized)   │           │
│  └─────────┘    └─────────────┘    └───────┬───────┘           │
│                                             │                   │
│  ┌─────────┐    ┌─────────────┐    ┌───────▼───────┐           │
│  │ SPI     │───►│ Weight      │◄───│ FFN /       │           │
│  │ Weight  │    │ SRAM        │    │ LayerNorm   │           │
│  │ Load    │    │ (4KB)       │    │             │           │
│  └─────────┘    └─────────────┘    └─────────────┘           │
└──────────────────────────────────────────────────────────────────┘
```

### SD Card Interface

The design implements a **1-bit SPI-mode SD card interface** (SD 2.0 compliant):

```
SD Card ──► [SD/MMC Controller] ──► [Data FIFO] ──► [Token Parser]
    │          (CMD/DAT lines)        (128 entries)    (token stream)
    │
    └── CMD: command line
    └── DAT[0]: data line (1-bit mode, lowest cost)
    └── CLK: card clock (25MHz max)
    └── VDD: card power (3.3V)
    └── VSS: ground
```

**Data Flow:**
1. SD card contains pre-tokenized text files (binary format: [sequence_length:4 bytes][token_ids...])
2. SD/MMC controller reads blocks via CMD/DAT lines
3. Data is buffered in on-chip FIFO (128 entries × 32-bit)
4. Token parser converts binary tokens to embedding indices
5. Embedding lookup feeds the transformer core

**Supported SD Card Features:**
- SD 2.0 specification (SPI mode, 1-bit data)
- Block addressing (512-byte blocks)
- Max clock: 25MHz (SD spec compliant)
- File format: simple binary (no FAT filesystem — custom format for minimal overhead)

### Transformer Core Design

#### Model Configuration

| Parameter | Value |
|-----------|-------|
| Transformer layers | 4 layers |
| Embedding dimension | 64 |
| Attention heads | 2 heads |
| FFN expansion | 4× (256-dim intermediate) |
| Sequence length | Up to 64 tokens |
| Vocabulary size | 512 tokens |
| Position encoding | Learnable embeddings (no RoPE) |

#### Quantization

| Layer Type | Weight Quantization | Activation Quantization |
|------------|---------------------|------------------------|
| Embedding | 8-bit | — |
| Attention WQ/WK/WV | 4-bit signed | 8-bit signed |
| Attention WO | 4-bit signed | 8-bit signed |
| FFN W1/W2 | 4-bit signed | 8-bit signed |
| LayerNorm gamma/beta | 8-bit signed | — |

**MAC Operation:**
- 4-bit weight × 8-bit activation → 12-bit partial product
- Accumulate with 20-bit precision
- Post-quantize to 8-bit for next layer

#### On-Chip Memory Map

```
┌──────────────────────────────────────────┐
│  Total On-Chip SRAM Budget: ~8KB        │
├──────────────────────────────────────────┤
│  Layer 0 Weights:  1.5KB                │
│  Layer 1 Weights:  1.5KB                │
│  Layer 2 Weights:  1.5KB                │
│  Layer 3 Weights:  1.5KB                │
│  Token FIFO:       512 bytes            │
│  Activation Buff:  1.5KB                │
│  Reserved:         ~512 bytes           │
├──────────────────────────────────────────┤
│  Weight Load: Via SPI before inference   │
│  (~6KB total weights, loaded in 12ms)   │
└──────────────────────────────────────────┘
```

### Streaming Inference Pipeline

```
[SD Card] → [FIFO] → [Embedding Lookup] → [Layer 0] → [Layer 1] → [Layer 2] → [Layer 3] → [Output]
              │                                         │                    │              │
          128-entry                               MAC arrays           MAC arrays      Softmax
          FIFO buffer                             (quantized)        (quantized)       (8-bit)
```

**Key Design: Layer-by-layer streaming**
- Only one layer's activations in memory at a time
- Weight SRAM holds ALL layer weights (pre-loaded via SPI)
- Input sequence is processed step-by-step (auto-regressive mode) or full sequence (inference mode)

---

## I/O Specification

| Signal | Direction | Description |
|--------|-----------|-------------|
| `clk` | Input | System clock (25MHz) |
| `rst_n` | Input | Active-low reset |
| `sd_cmd` | Bidir | SD card command line |
| `sd_dat` | Input | SD card data line (1-bit) |
| `sd_clk` | Output | SD card clock (25MHz) |
| `spi_sclk` | Input | SPI clock for host |
| `spi_cs_n` | Input | SPI chip select (active low) |
| `spi_mosi` | Input | SPI data input (weight load / control) |
| `spi_miso` | Output | SPI data output (prediction result) |
| `irq_done` | Output | Inference complete interrupt |
| `vdd` | Power | Core supply (1.8V) |
| `vss` | Ground | Core ground |
| `io_vdd` | Power | I/O supply (5.0V tolerant) |
| `io_vss` | Ground | I/O ground |
| `sd_vdd` | Power | SD card power (3.3V) |
| `sd_gnd` | Ground | SD card ground |

**Total pins: 16**

---

## SD Card Data Format

### File Format (Binary)

```
┌────────────────────────────────────────────────────┐
│  SD Card File Layout                               │
├────────────────────────────────────────────────────┤
│  [Header: 8 bytes]                                 │
│    Magic: 0x5452414E ("TRAN")                      │
│    Version: 1 (uint32)                             │
│    Num Sequences: N (uint16)                       │
│    Seq Length: L (uint16)                          │
├────────────────────────────────────────────────────┤
│  [Sequence Data: N × L × 2 bytes]                  │
│    Sequence 0: [token_0][token_1]...[token_{L-1}]  │
│    Sequence 1: [token_0][token_1]...[token_{L-1}]  │
│    ...                                            │
├────────────────────────────────────────────────────┤
│  [Labels: N × 1 byte (optional)]                   │
│    Label_0, Label_1, ...                           │
│    Used for classification tasks                   │
└────────────────────────────────────────────────────┘
```

**Example (512-byte SD block):**
```
Header (8 bytes):  54 52 41 4E 01 00 00 00
Sequence data:     [token IDs in 16-bit big-endian]
Total per seq:     64 tokens × 2 bytes = 128 bytes
Block alignment:   Each sequence padded to 512-byte SD block boundary
```

### Host Tool (Python)

A simple Python tool on the host PC converts text to the binary format:

```python
# Host tool: sdcard_writer.py
# Usage: python3 sdcard_writer.py input_text.bin "hello world..."

# Features:
# 1. Tokenize text using a shared vocabulary (512 tokens)
# 2. Pad/trim sequences to fixed length
# 3. Write binary format for SD card
# 4. Generate weight file for SPI loading
```

---

## Performance Targets

| Metric | Target |
|--------|--------|
| Area | < 400um × 400um |
| Clock | 25MHz (SD card limit) |
| Inference (full seq) | < 2ms |
| SD card read speed | ~25KB/s (1-bit mode) |
| Power (active) | < 2mW |
| Weight capacity | ~6KB SRAM |
| Seq length | Up to 64 tokens |
| Vocabulary | 512 tokens |
| Layers | 4 transformer layers |

---

## Implementation Plan

### Phase 1: SD Card Controller & Tokenizer (Weeks 1-3)
- Design SD/MMC SPI-mode controller
- Implement command/response parser
- Design token FIFO and parser
- **Deliverable**: SD card controller RTL + testbench

### Phase 2: Transformer Core (Weeks 4-7)
- Implement quantized attention engine (2 heads, 64-dim)
- Design FFN and LayerNorm modules
- Create streaming data path between layers
- Weight SRAM controller (SPI load)
- **Deliverable**: Transformer core RTL + simulation

### Phase 3: System Integration (Weeks 8-9)
- Integrate SD controller + transformer core + output serializer
- Design control FSM (SD read → tokenize → inference → output)
- SPI weight load protocol
- **Deliverable**: Full chip RTL + co-simulation

### Phase 4: Verification & Co-simulation (Weeks 10-11)
- Python-trained transformer model → RTL co-simulation
- End-to-end test: text → SD card file → hardware inference
- Quantization accuracy analysis
- **Deliverable**: Verification report

### Phase 5: Synthesis & Layout (Weeks 12-14)
- Synthesis with Yosys + gf180mcu library
- STA with OpenSTA
- Layout of transformer core
- DRC/LVS verification
- **Deliverable**: GDSII + signoff reports

### Phase 6: Final Deliverables (Weeks 15-16)
- Full integration test
- Host tool for SD card file creation
- Complete documentation
- **Deliverable**: Final GDSII + host tools + docs

---

## Host Toolchain

### Software Tools on PC

1. **`sdcard_writer.py`**: Text → SD card binary format
2. **`weight_loader.py`**: Trained model weights → SPI binary format
3. **`tokenizer.py`**: Shared vocabulary (512 tokens) for text/token conversion

### Training Pipeline

```
Raw text → Tokenizer (512 vocab) → Transformer training (Python/PyTorch)
                                              │
                                              ├── Export weights → SPI binary
                                              └── Export test sequences → SD card binary
```

### Quantization Aware Training (QAT)

1. Train transformer in PyTorch with standard precision
2. Apply Quantization Aware Training (QAT) for 4-bit/8-bit
3. Validate accuracy degradation (< 3% on target task)
4. Export quantized weights

---

## Use Cases

### 1. Keyword Spotting
- SD card contains audio-derived token sequences
- Transformer classifies: ON/OFF/START/STOP/ERROR
- Output: class probability via SPI

### 2. Simple Chat Bot (Limited Vocabulary)
- SD card contains conversation context (up to 64 tokens)
- Transformer generates next token (up to 10 tokens)
- Each token output via SPI

### 3. Sensor Anomaly Detection
- SD card contains sensor reading sequences
- Transformer trained to detect normal vs anomaly patterns
- Output: anomaly probability via SPI

### 4. Language Model for Specific Domain
- SD card contains domain-specific text (medical, legal, etc.)
- Transformer predicts next word from vocabulary
- Useful for low-power embedded text completion

---

## Resource Requirements

- **GF180 MCU 7T/9T** standard cell library
- **gf180mcuD** SRAM macros (for weight and activation storage)
- **LibreLane** RTL-to-GDS flow
- **XSCHEM + ngspice** for SD card interface verification
- **Yosys + OpenROAD** for digital synthesis and layout
- **OpenSTA** for timing analysis
- **Python/PyTorch** for model training and QAT

---

## Differentiation from Existing Projects

| Feature | #135 (LSTM OCR) | #106 (AI_Byte Accelerator) | This Proposal |
|---------|-----------------|---------------------------|---------------|
| Architecture | LSTM | Reconfigurable MAC | Transformer |
| Input source | SPI | SPI | **SD card** |
| Data format | Feature vectors | Generic | **Tokenized text** |
| Task | OCR | Generic AI | **NLP / text** |
| Area | ~2mm² | ~1mm² | < 0.16mm² |
| On-chip memory | Weight SRAM | MAC array | Weight + activation SRAM |
| SD interface | No | No | **Yes** |
| Edge deployment | SPI only | SPI only | **SD card + SPI** |
| Sequence length | Single vector | Single vector | **Up to 64 tokens** |
| Task variety | OCR only | General | **Keyword, chat, anomaly** |

---

## Risk Mitigation

| Risk | Mitigation |
|------|-----------|
| SD card interface complexity | Use 1-bit SPI mode (simplest) |
| Transformer area too large | Reduce layers to 2, heads to 1 |
| Quantization accuracy loss | Use QAT, validate on target task |
| SD card speed bottleneck | Pipeline SD read with compute |
| Weight SRAM insufficient | Stream layers (load weights per layer) |

---

## References

- Vaswani, A. et al. (2017). "Attention Is All You Need"
- SD Physical Layer Simplified Specification (SD Association)
- Liao, M. et al. (2021). "A 1.5pJ/Step Spiking LSTM Inference Engine in 28nm"
- Han, S. et al. (2016). "EIE: Efficient Inference Engine on Compressed Deep Neural Network"
- GF180 MCU PDK documentation
- LibreLane RTL-to-GDS flow documentation

---

*Last updated: 2026-06-08*