Blog posts

Star Attention: Efficient LLM Inference Over Long Sequences

2 minute read

Published: December 04, 2024

• Abstract
• Introduction
  • Phase 1: Context Encoding
  • Phase 2: Query Encoding and Token Generation
  • Update of Cache and Output
• Thoughts

Preface

1 minute read

Published: November 27, 2024

I am on a train from Beijing to Wuxin right now. Sometimes I believe one’s life is just like traveling on a train…