Test Your Transformers Knowledge: ML Practice Quiz
Do you think you know everything about Transformers and Machine Learning? Put your knowledge to the test with our practice quiz based on the attention mechanism in the 'Attention is All You Need' paper!
Question 1 of 20Test Your Transformers Knowledge: ML Practice Quiz
All 20 questions
- What kind of model is the Transformer?A model architecture eschewing recurrence · A recurrent neural network model
- How many parallel attention layers, or heads, does the Transformer use?8 · 16
- What is the benefit of multi-head attention?Allows the model to jointly attend to information from different representation subspaces · To perform a single attention function more efficiently
- Which tasks has self-attention been used successfully in?Reading comprehension, abstractive summarization, textual entailment · Image classification, video analysis, speech recognition
- In encoder-decoder attention layers, where do the queries come from?The previous decoder layer · The initial input sequence
- What tasks have end-to-end memory networks performed well in?Simple-language question answering and language modeling · Image classification and object detection
- What is the value of dk and dv in the Transformer's attention heads?64 · 32
- What transformer application issue does the local, restricted attention mechanism aim to handle efficiently?Large inputs and outputs such as images, audio and video · Small and simple text sequences
- How many GPUs were used to train the Transformer in the experiment mentioned?8 P100 GPUs · 4 P100 GPUs
- What sequence transduction model comparison shows the superiority of the Transformer?WMT 2014 English-to-German translation task · WMT 2016 English-to-Japanese translation task
- How does the Transformer handle sequence modeling differently compared to RNNs?It relies entirely on an attention mechanism · It uses LSTM cells
- What is the final result of the multi-head attention's output values?They are concatenated and projected · They are averaged
- What BLEU score does the Transformer achieve on the WMT 2014 English-to-French translation task?41.8 · 35.6
- What is the main advantage of the Transformer over RNNs and CNNs?More parallelizable and requires less training time · Higher accuracy in image classification
- Which model's performance does the Transformer surpass on the WSJ training set of 40K sentences?Berkeley-Parser · Stanford-Parser
- What are the key dependencies handled by multi-head attention when modeling sentences?Different representation subspaces at different positions · Time series dependencies
- What is the central computation unit in the Transformer model?Scaled dot-product attention and multi-head attention · Convolutions and pooling layers
- In encoder-decoder attention, from where do the memory keys and values come?The output of the encoder · Previous attention heads
- Which layer's attention heads are shown in Figure 3 to follow long-distance dependencies?Layer 5 of 6 · Layer 6 of 6
- What kind of training time advantage does the Transformer have?Significantly faster than RNN/CNN-based architectures · Significantly slower





