📖 Tier 1: Prepare & Study Guide ✓ 100% Solved with Rationales

Transformers, LLMs & Natural Language Processing (Artificial Intelligence) Solved Questions & Notes (2026) - Apex Rankers

Artificial Intelligence & Data Science > Artificial Intelligence > Transformers, LLMs & Natural Language Processing

82 Total Solved Questions
~123 mins Estimated Reading Time
1 Subject Areas / Chapters
Select Topic Area / Chapter: Click any section below to switch questions

Transformers, LLMs & Natural Language Processing

100%
Showing 25 of 82 (30%)
🎯 Practice
Jump:
Q. 1 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is the core mathematical formula for Scaled Dot-Product Attention in the original Transformer ('Attention Is All You Need', Vaswani et al. 2017)?
A
Attention(Q, K, V) = softmax((Q * K^T) / sqrt(d_k)) * V
✓ Correct
B
Attention(Q, K, V) = sigmoid(Q * K + V)
C
Attention(Q, K, V) = tanh((Q + K) * V)
D
Attention(Q, K, V) = ReLU(Q * K^T) * V
💡 Step-by-Step Explanation & Concept Rationale
Dividing by sqrt(d_k) prevents dot products from growing excessively large for large dimensions, which would otherwise push the softmax into regions with vanishing gradients.
Q. 2 Artificial Intelligence
Difficulty: Hard (1 Mark)
Why is the scaling factor 1 / sqrt(d_k) crucial in Scaled Dot-Product Attention?
A
For large key dimensions d_k, the dot products grow large in magnitude, pushing the softmax function into regions with extremely small gradients
✓ Correct
B
It reduces matrix memory size on GPU
C
It converts complex numbers to real floats
D
It bounds output values strictly between -1 and 1
💡 Step-by-Step Explanation & Concept Rationale
Under independent zero-mean unit-variance components, the dot product has mean 0 and variance d_k; dividing by sqrt(d_k) restores unit variance.
Q. 3 Artificial Intelligence
Difficulty: Medium (1 Mark)
In Multi-Head Attention, why are multiple projection heads used rather than a single large attention head?
A
Multiple heads allow the model to jointly attend to information from different representation subspaces at different positions simultaneously
✓ Correct
B
Multiple heads allow the model to train on multiple GPUs without code changes
C
Multiple heads reduce the total number of matrix multiplications
D
Multiple heads eliminate the need for feed-forward layers
💡 Step-by-Step Explanation & Concept Rationale
Multi-Head Attention: Concat(head_1, ..., head_h) * W_O, where each head can capture distinct syntactic and semantic relationships.
Q. 4 Artificial Intelligence
Difficulty: Medium (1 Mark)
Why are 'Positional Encodings' strictly necessary in Transformer architectures?
A
Because the self-attention mechanism is inherently permutation-equivariant (it has no built-in notion of token order or sequence position)
✓ Correct
B
To compress long sentences into short embeddings
C
To translate text between different natural languages
D
To prevent integer overflow during softmax calculation
💡 Step-by-Step Explanation & Concept Rationale
Unlike RNNs which process tokens step-by-step, Transformers process all tokens in parallel; without positional encodings, word order is lost.
Q. 5 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'Byte-Pair Encoding' (BPE) tokenization widely used in GPT models?
A
A subword tokenization algorithm that iteratively merges the most frequent pair of adjacent bytes or characters in a text corpus into a new vocabulary token
✓ Correct
B
An encryption algorithm that secures text against cyberattacks
C
A method that converts text into raw ASCII binary bits
D
A tokenization method that splits text strictly on whitespaces
💡 Step-by-Step Explanation & Concept Rationale
BPE balances word-level semantics with character-level coverage, eliminating Out-Of-Vocabulary (OOV) tokens by decomposing rare words into subword pieces.
Q. 6 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is 'WordPiece' tokenization (used in BERT)?
A
A subword tokenization algorithm that selects symbol pairs to merge based on maximizing the likelihood of the training language model rather than pure frequency count
✓ Correct
B
A tool for checking spelling mistakes in Microsoft Word
C
A method that limits vocabularies to 1,000 words only
D
A syntax parser that extracts noun phrases
💡 Step-by-Step Explanation & Concept Rationale
WordPiece uses the '##' prefix for subword continuations (e.g. 'playing' -> 'play', '##ing') and optimizes corpus likelihood.
Q. 7 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'SentencePiece' tokenization (used in T5, LLaMA)?
A
A language-independent subword tokenizer that treats raw input text as a stream of Unicode characters including whitespace (represented as '_'), without requiring language-specific pre-tokenizers
✓ Correct
B
A model that generates full sentences from single words
C
A grammar checking rule engine
D
A tool that counts the number of sentences in a document
💡 Step-by-Step Explanation & Concept Rationale
SentencePiece enables seamless multi-lingual tokenization without language-specific regex or whitespace assumptions.
Q. 8 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is the fundamental architectural difference between BERT and GPT?
A
BERT is an Encoder-only model trained with bidirectional masked language modeling; GPT is a Decoder-only autoregressive model trained with causal left-to-right next-token prediction
✓ Correct
B
BERT is for images; GPT is for audio
C
BERT has no attention layers; GPT uses only attention layers
D
BERT runs only on CPUs; GPT runs on TPUs
💡 Step-by-Step Explanation & Concept Rationale
BERT excels at representation and comprehension (classification, NER, QA); GPT excels at generative text synthesis.
Q. 9 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is 'Masked Language Modeling' (MLM) pre-training objective introduced by BERT?
A
Randomly masking 15% of input tokens with a [MASK] token and training the bidirectional encoder to predict the original identity of the masked words from context
✓ Correct
B
Hiding user personal data for privacy
C
Masking convolutional feature maps with zero padding
D
Filtering out offensive words from internet text
💡 Step-by-Step Explanation & Concept Rationale
MLM enables deep bidirectional representation learning by forcing the model to fuse left and right context representations simultaneously.
Q. 10 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'Causal Masking' (Autoregressive Masking / Look-Ahead Mask) in Transformer Decoders (e.g. GPT)?
A
An upper-triangular attention mask that sets future token attention scores to -infinity before softmax, ensuring position i can only attend to positions <= i
✓ Correct
B
A mask that prevents models from generating offensive text
C
A mask that hides images from text decoders
D
A technique that deletes punctuation tokens
💡 Step-by-Step Explanation & Concept Rationale
Causal masking enforces the autoregressive property during parallel training: token t cannot cheat by looking at ground-truth future token t+1.
Q. 11 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'RoBERTa' (Robustly Optimized BERT Approach) and what key modifications did it make to BERT?
A
Trained longer on larger batches with more data, removed the Next Sentence Prediction (NSP) task, used dynamic masking, and adopted a larger byte-level BPE vocabulary
✓ Correct
B
Replaced all self-attention with convolutional filters
C
Reduced the number of layers to 2
D
Added a robotic text-to-speech module
💡 Step-by-Step Explanation & Concept Rationale
Liu et al. proved BERT was significantly undertrained; removing NSP and scaling data/batch size dramatically boosted GLUE benchmark scores.
Q. 12 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is the 'T5' (Text-to-Text Transfer Transformer) framework introduced by Google?
A
A unified Encoder-Decoder model that casts every NLP problem (classification, summarization, translation, QA) into a text-to-text format
✓ Correct
B
A 5-layer convolutional vision model
C
A programming language for training neural nets
D
A tool for converting text into SQL databases
💡 Step-by-Step Explanation & Concept Rationale
T5 uses prefix prompts (e.g. 'translate English to German: ...') to handle diverse NLP tasks within a single unified architecture.
Q. 13 Artificial Intelligence
Difficulty: Medium (1 Mark)
In Retrieval-Augmented Generation (RAG), what is the primary workflow?
A
1. Chunk and embed private documents into a Vector Database, 2. Retrieve top-k relevant chunks based on user query similarity, 3. Augment the LLM prompt with retrieved context for grounded generation
✓ Correct
B
1. Retrain the entire LLM from scratch on private data every day
C
1. Randomly guess relevant facts from Wikipedia
D
1. Convert text documents into audio files
💡 Step-by-Step Explanation & Concept Rationale
RAG overcomes LLM hallucinations and knowledge cutoff dates by dynamically retrieving factual domain context at inference time.
Q. 14 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is a 'Vector Database' (e.g. Pinecone, Milvus, Qdrant, Chroma, pgvector) optimized for?
A
High-throughput, low-latency Approximate Nearest Neighbor (ANN) search over high-dimensional dense embedding vectors (e.g. HNSW, IVF-PQ)
✓ Correct
B
Storing relational SQL tables with primary keys only
C
Storing raw video files for streaming
D
Compiling C++ programs into machine code
💡 Step-by-Step Explanation & Concept Rationale
Vector databases index dense vector embeddings to perform sub-millisecond similarity queries across millions of multi-dimensional vectors.
Q. 15 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'HNSW' (Hierarchical Navigable Small World) in vector search indexing?
A
A multi-layer graph-based index for fast Approximate Nearest Neighbor (ANN) search with logarithmic O(log N) search complexity
✓ Correct
B
A hardware acceleration chip for GPUs
C
A network security firewall protocol
D
A loss function for training BERT models
💡 Step-by-Step Explanation & Concept Rationale
HNSW builds a hierarchy of proximity graphs where upper layers enable fast coarse routing and bottom layers execute precise local neighbor search.
Q. 16 Artificial Intelligence
Difficulty: Hard (1 Mark)
In dense text retrieval, what is 'Bi-Encoder' versus 'Cross-Encoder' architecture?
A
Bi-Encoders encode query and document independently into vectors for fast vector DB search; Cross-Encoders process (query, doc) pairs jointly through full self-attention for higher accuracy reranking
✓ Correct
B
Bi-Encoders use two GPUs; Cross-Encoders use one GPU
C
Bi-Encoders are for images; Cross-Encoders are for audio
D
There is no difference between them
💡 Step-by-Step Explanation & Concept Rationale
Bi-encoders (e.g. sentence-transformers) enable sub-millisecond vector retrieval; cross-encoders are used as second-stage rerankers on top-100 candidates.
Q. 17 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'QLoRA' (Quantized Low-Rank Adaptation) and how does it enable fine-tuning a 65B LLM on a single 48GB GPU?
A
Quantizing the base model weights to 4-bit NormalFloat (NF4), using Double Quantization to compress quantization constants, and applying Paged Optimizers to manage memory spikes during LoRA fine-tuning
✓ Correct
B
Deleting 75% of the base model weights permanently
C
Training only the first 2 layers of the network
D
Replacing all attention layers with linear regression
💡 Step-by-Step Explanation & Concept Rationale
Dettmers et al. proved QLoRA matches full 16-bit fine-tuning performance while reducing required GPU VRAM from >780GB to <48GB.
Q. 18 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'RLHF' (Reinforcement Learning from Human Feedback) in LLM alignment?
A
1. Train a Reward Model on human preference pairwise comparisons, 2. Fine-tune the base policy LLM using PPO (Proximal Policy Optimization) against the reward model with a KL penalty
✓ Correct
B
Asking humans to write all LLM responses manually in real-time
C
Training models by punishing developers when errors occur
D
A game where humans play chess against AI models
💡 Step-by-Step Explanation & Concept Rationale
RLHF aligns raw pre-trained LLMs to be helpful, honest, and harmless (HHH) based on human preference signals.
Q. 19 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'DPO' (Direct Preference Optimization) and how does it simplify LLM alignment compared to RLHF with PPO?
A
It mathematically reparameterizes the reward model directly in terms of the language model policy, optimizing preference data with a simple closed-form cross-entropy loss without training a separate reward model or using unstable RL (PPO)
✓ Correct
B
It replaces text tokens with numerical integers
C
It trains models with direct supervised classification labels only
D
It uses genetic evolution instead of neural nets
💡 Step-by-Step Explanation & Concept Rationale
Rafailov et al. showed DPO eliminates the complexity and instability of PPO while achieving equal or superior alignment performance.
Q. 20 Artificial Intelligence
Difficulty: Medium (1 Mark)
In prompt engineering, what is 'Chain-of-Thought' (CoT) prompting (Wei et al. 2022)?
A
Prompting the LLM to generate explicit intermediate reasoning steps before arriving at the final answer ('Let's think step by step'), dramatically improving multi-step mathematical and logical reasoning
✓ Correct
B
Connecting multiple LLMs with physical cables
C
Sending prompts in a continuous loop to overload the server
D
Translating prompts through 5 different languages
💡 Step-by-Step Explanation & Concept Rationale
CoT allocates more test-time computation to intermediate token generation, allowing models to decompose complex reasoning problems.
Q. 21 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is 'Self-Consistency' in Chain-of-Thought prompting?
A
Sampling multiple diverse reasoning paths from the LLM with temperature > 0 and selecting the final answer via majority vote across all sampled paths
✓ Correct
B
Ensuring the LLM never changes its answer
C
Validating Python code syntax
D
Checking that prompt words are spelled correctly
💡 Step-by-Step Explanation & Concept Rationale
Wang et al. showed self-consistency over multiple reasoning trajectories delivers significant reasoning accuracy gains.
Q. 22 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'ReAct' (Reasoning + Acting) framework for LLM autonomous agents?
A
An agentic paradigm that interleaves reasoning traces ('Thought'), action execution ('Action' via external tools/APIs), and environmental feedback ('Observation')
✓ Correct
B
A JavaScript library for building web UIs
C
A chemical reaction simulation engine
D
A reactive programming library for mobile apps
💡 Step-by-Step Explanation & Concept Rationale
Yao et al. demonstrated that combining reasoning with external tool usage (search, Python execution, SQL) enables robust problem solving.
Q. 23 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is 'Tool Calling' (Function Calling) in modern LLM APIs?
A
The capability of an LLM to detect when an external function should be called, outputting structured JSON arguments matching a predefined schema rather than plain text
✓ Correct
B
Making telephone calls using voice AI
C
Calling Python functions manually from terminal
D
Installing development tools on Linux
💡 Step-by-Step Explanation & Concept Rationale
Function calling enables deterministic integration between LLMs and databases, APIs, calculation engines, and ERP systems.
Q. 24 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is 'Hallucination' in Large Language Models and what are its primary causes?
A
The generation of factually incorrect, fabricated, or nonsensical assertions presented with high confidence, caused by statistical pattern completion over noisy training data and lack of grounded world state
✓ Correct
B
A visual display glitch on GPU monitors
C
A virus infecting the LLM server
D
A power outage during inference
💡 Step-by-Step Explanation & Concept Rationale
LLMs optimize for plausible sequence continuation rather than absolute factual truth; mitigated via RAG, citation grounding, and guardrails.
Q. 25 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is 'BLEU' (Bilingual Evaluation Understudy) score and what is its primary use?
A
A metric measuring the n-gram overlap precision between machine-generated translations and human reference translations (with a brevity penalty for short outputs)
✓ Correct
B
A metric measuring image brightness
C
A test for measuring internet bandwidth
D
A tool for grading spoken pronunciation
💡 Step-by-Step Explanation & Concept Rationale
BLEU computes geometric average of n-gram precisions (n=1 to 4) multiplied by brevity penalty BP.
Study Stream Progress: Showing 25 of 82 Questions (30%)
Jump to:

Ready to Test Your Retention & Speed?

Now that you have reviewed the study questions and rationales, test yourself in our interactive 1-by-1 practice engine or take the full official timed mock exam.