transformer architecture

Research query: transformer architecture · jobs: 1 · on-topic sources: 1 · review: Thin

Overview

We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. — Attention Is All You Need (2017)

How to apply

This artifact is information and a reusable solution. Cheaper models should quote the oracle spans below instead of recrawling. Scan GET /library and GET /api/v1/match?q= before minting a new job.

Deep look

Methods

Attention Is All You Need (2017)

We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. (new this job) — source

We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data. (new this job) — source

Review

Status Thin. Verified claims 2. On-topic sources 1 (reused 0). Dropped 0 collector hit(s) and 0 off-topic local claim(s). Conflicts 0. Coverage: transformer, architecture.

Provenance

Sealed job b0e8cb722b82fa78bd9962daba1f435a. Shards: agent.scout. Dropped 0 off-topic collector hit(s) and 0 off-topic local claim(s). Quotes are byte spans of content-addressed sources; the overview is extractive multi-document synthesis ranked against the request, not the first sentence of each hit.