TII didn't just use FlashAttention v2; they forked it. Inside the falcon/cuda directory, there are custom fused kernels that merge the residual add, layer norm, and attention output into a single kernel launch. The comment in the code reads: "// Merged to overcome memory bandwidth bottleneck on A100-40GB"
When Falcon 40B was released, its "exclusive" nature was defined by two major deviations from the standard LLaMA architecture established by Meta: falcon 40 source code exclusive
: Users must own a licensed copy of the original 1998 game to run BMS, which serves as a "check" for legal compliance. The 2025/2026 Legacy: Falcon 4.38 Source Code - Falcon 4 history TII didn't just use FlashAttention v2; they forked it