📖 Tier 1: Prepare & Study Guide ✓ 100% Solved with Rationales

Deep Learning Architectures & Optimization (Artificial Intelligence) Solved Questions & Notes (2026) - Apex Rankers

Artificial Intelligence & Data Science > Artificial Intelligence > Deep Learning Architectures & Optimization

82 Total Solved Questions
~123 mins Estimated Reading Time
1 Subject Areas / Chapters
Select Topic Area / Chapter: Click any section below to switch questions

Deep Learning Architectures & Optimization

100%
Showing 25 of 82 (30%)
🎯 Practice
Jump:
Q. 1 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is the primary cause of the 'Vanishing Gradient Problem' during backpropagation in deep neural networks when using Sigmoid or Tanh activation functions?
A
The derivatives of Sigmoid and Tanh are strictly less than 1 (maximum 0.25 for Sigmoid), causing error gradients to shrink exponentially as they are multiplied through multiple layers
✓ Correct
B
The learning rate is set too high, causing weights to oscillate to infinity
C
The dataset contains negative numbers only
D
The GPU runs out of video memory during matrix multiplication
💡 Step-by-Step Explanation & Concept Rationale
By the chain rule, multiplying numbers < 1 across many layers causes gradients for early layers to vanish toward zero, halting weight updates.
Q. 2 Artificial Intelligence
Difficulty: Medium (1 Mark)
How does the Rectified Linear Unit (ReLU: f(x) = max(0, x)) activation function mitigate the vanishing gradient problem in deep networks?
A
For all positive inputs (x > 0), the gradient is constant at 1.0, allowing error signals to flow backward through deep layers without exponential decay
✓ Correct
B
It bounds outputs between -1 and +1
C
It uses exponential functions to amplify gradients
D
It calculates second-order derivatives automatically
💡 Step-by-Step Explanation & Concept Rationale
ReLU's constant gradient of 1 for x > 0 eliminates gradient saturation in the positive regime.
Q. 3 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is the 'Dying ReLU' problem and what architectural modification directly addresses it?
A
Neurons whose inputs are consistently negative output 0 with 0 gradient and permanently stop learning; addressed by Leaky ReLU (f(x) = max(alpha * x, x)) or PReLU
✓ Correct
B
Neurons overheating the GPU processor; addressed by liquid cooling
C
Weights becoming NaN due to division by zero; addressed by adding epsilon
D
Loss function diverging to infinity; addressed by weight decay
💡 Step-by-Step Explanation & Concept Rationale
Leaky ReLU assigns a small non-zero slope (e.g. 0.01) for x < 0, allowing gradients to flow and revive dead neurons.
Q. 4 Artificial Intelligence
Difficulty: Hard (1 Mark)
What activation function, defined as x * Phi(x) where Phi is the standard Gaussian cumulative distribution function, is widely used in modern Transformers like BERT and GPT?
A
Gaussian Error Linear Unit (GELU)
✓ Correct
B
Sigmoid
C
Hard Tanh
D
Binary Step Function
💡 Step-by-Step Explanation & Concept Rationale
GELU weights inputs by their probability under a Gaussian distribution, providing smooth non-linear probabilistic gating.
Q. 5 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is the fundamental purpose of 'Backpropagation' in artificial neural networks?
A
Efficiently computing the partial derivatives (gradients) of the loss function with respect to every learnable weight in the network using the calculus Chain Rule
✓ Correct
B
Sorting training data in ascending order
C
Generating random initial weights for the network
D
Converting images into grayscale matrices
💡 Step-by-Step Explanation & Concept Rationale
Backpropagation propagates errors backward from the output layer to compute dLoss/dWeight for gradient descent optimization.
Q. 6 Artificial Intelligence
Difficulty: Hard (1 Mark)
In stochastic gradient descent, what is the role of the 'Momentum' term?
A
Accelerating gradient descent in the relevant direction and dampening oscillations by accumulating an exponentially decaying moving average of past gradients
✓ Correct
B
Randomly resetting weights to zero every epoch
C
Increasing the batch size dynamically
D
Normalizing input pixel values
💡 Step-by-Step Explanation & Concept Rationale
Momentum v_t = gamma * v_{t-1} + eta * grad helps the optimizer navigate valleys and push past flat local minima or saddle points.
Q. 7 Artificial Intelligence
Difficulty: Hard (1 Mark)
How does the 'Adam' (Adaptive Moment Estimation) optimizer compute parameter updates?
A
By maintaining exponentially decaying moving averages of both past gradients (first moment / mean) and past squared gradients (second moment / uncentered variance)
✓ Correct
B
By computing exact second-order Hessian matrices at every step
C
By using random walk exploration without gradients
D
By keeping learning rates completely constant for all weights
💡 Step-by-Step Explanation & Concept Rationale
Adam combines the benefits of AdaGrad (handling sparse gradients) and RMSprop (handling non-stationary objectives) with bias-corrected moment estimates.
Q. 8 Artificial Intelligence
Difficulty: Hard (1 Mark)
Why is 'AdamW' (Adam with Decoupled Weight Decay) preferred over standard Adam with L2 regularization in modern deep learning?
A
Standard Adam ties L2 regularization to gradient magnitude adaptation, whereas AdamW decouples weight decay, applying it directly to weights and restoring true regularization behavior
✓ Correct
B
AdamW runs twice as fast on TPU hardware
C
AdamW eliminates the need for learning rates
D
AdamW works only on convolutional layers
💡 Step-by-Step Explanation & Concept Rationale
Loshchilov & Hutter showed that in Adam, L2 regularization is distorted by moving average scaling; decoupling weight decay significantly improves generalization.
Q. 9 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'Batch Normalization' (BatchNorm) and what are its primary benefits during deep network training?
A
Normalizing layer inputs across the mini-batch to zero mean and unit variance, which stabilizes internal covariate shift, accelerates training, and acts as a mild regularizer
✓ Correct
B
Normalizing the total number of training epochs
C
Rounding all weights to 8-bit integers
D
Scaling the learning rate by the number of GPUs
💡 Step-by-Step Explanation & Concept Rationale
BatchNorm adds learnable scale (gamma) and shift (beta) parameters, enabling higher learning rates and reducing sensitivity to weight initialization.
Q. 10 Artificial Intelligence
Difficulty: Hard (1 Mark)
Why is 'Layer Normalization' (LayerNorm) universally preferred over Batch Normalization in Transformers and Recurrent Neural Networks (RNNs)?
A
LayerNorm computes mean and variance across the feature/channel dimension for each individual sample independently of batch size, making it ideal for variable-length sequences and batch size = 1
✓ Correct
B
LayerNorm uses GPU shared memory while BatchNorm uses disk storage
C
LayerNorm requires no floating-point math
D
LayerNorm works only on 2D images
💡 Step-by-Step Explanation & Concept Rationale
BatchNorm depends on mini-batch statistics, failing with small batch sizes or dynamic sequence lengths; LayerNorm normalizes across hidden dimensions per token.
Q. 11 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is the mechanism and regularization effect of 'Dropout' during neural network training?
A
Randomly setting a fraction p of neuron activations to zero during each forward pass, preventing complex co-adaptations of feature detectors
✓ Correct
B
Permanently deleting 50% of the layers from the model file
C
Dropping slow internet connection packets during distributed training
D
Stopping training early when loss reaches a threshold
💡 Step-by-Step Explanation & Concept Rationale
Dropout forces the network to learn robust, redundant representations by effectively sampling a thinned ensemble of sub-networks.
Q. 12 Artificial Intelligence
Difficulty: Hard (1 Mark)
During inference/testing, how is Dropout handled?
A
Dropout is disabled (all neurons are active), and activations are scaled by (1 - p) (or inverted dropout scales by 1/(1-p) during training) to preserve expected output magnitude
✓ Correct
B
Dropout rate is increased to 90%
C
All weights are randomly permuted
D
The network computes outputs using only 1 neuron
💡 Step-by-Step Explanation & Concept Rationale
Inverted dropout scales activations during training by 1/(1-p) so that standard unscaled forward passes can execute at test time.
Q. 13 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'He (Kaiming) Initialization' and why is it recommended for neural networks using ReLU activations?
A
Drawing weights from a Gaussian distribution with variance 2 / n_in to keep the variance of activations and backpropagated gradients constant across layers with half-rectified activations
✓ Correct
B
Initializing all weights to exactly 1.0
C
Setting all weights to zero
D
Drawing weights from a uniform distribution between -100 and +100
💡 Step-by-Step Explanation & Concept Rationale
Because ReLU zeroes out negative inputs (halving variance), Kaiming initialization uses a factor of 2 (unlike Xavier/Glorot's 1/n) to prevent signal attenuation.
Q. 14 Artificial Intelligence
Difficulty: Medium (1 Mark)
What happens if all weights in a multi-layer neural network are initialized to identical constant values (e.g. all zeros or all ones)?
A
All hidden units in a layer compute identical outputs and receive identical gradients, preventing the network from breaking symmetry and learning distinct features
✓ Correct
B
The network trains at maximum theoretical speed
C
The loss function immediately converges to zero
D
The model becomes immune to overfitting
💡 Step-by-Step Explanation & Concept Rationale
Symmetry breaking requires random weight initialization so that different neurons track different input features.
Q. 15 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is the core mathematical operation performed by a 2D Convolutional layer in a CNN?
A
Element-wise multiplication of a learnable filter/kernel with localized receptive field patches followed by summation and addition of a bias term
✓ Correct
B
Matrix inversion of the full image matrix
C
Sorting pixels in ascending brightness order
D
Computing the determinant of image RGB channels
💡 Step-by-Step Explanation & Concept Rationale
Convolution exploits spatial locality and translation invariance through shared filter weights sliding across input feature maps.
Q. 16 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is 'Stride' in a convolutional layer?
A
The step size (number of pixels) by which the convolutional filter shifts horizontally and vertically across the input feature map
✓ Correct
B
The thickness of the input image border padding
C
The number of color channels in the image
D
The total number of filters in the layer
💡 Step-by-Step Explanation & Concept Rationale
A stride of 1 shifts the kernel 1 pixel at a time; a stride of 2 skips every other pixel, halving spatial output dimensions (downsampling).
Q. 17 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is 'Padding' (e.g. 'same' padding) in CNNs?
A
Adding artificial pixels (typically zeros) around the spatial borders of an input feature map to preserve spatial dimensions after convolution and retain border information
✓ Correct
B
Compressing image files into JPEG format
C
Normalizing pixel values between 0 and 1
D
Deleting dark pixels from image edges
💡 Step-by-Step Explanation & Concept Rationale
'Same' padding ensures the output spatial dimensions equal the input spatial dimensions (when stride=1); 'valid' padding performs no padding.
Q. 18 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is the purpose of a 'Max Pooling' layer in CNN architectures?
A
Downsampling spatial dimensions (height and width) of feature maps, reducing computational parameters while providing spatial translation invariance to minor distortions
✓ Correct
B
Increasing the number of color channels
C
Learning non-linear weights via backpropagation
D
Normalizing pixel brightness across the batch
💡 Step-by-Step Explanation & Concept Rationale
Max pooling extracts the maximum value in local windows (e.g. 2x2 with stride 2), retaining dominant features while halving resolution.
Q. 19 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'Global Average Pooling' (GAP) and why is it used to replace dense fully-connected layers at the end of modern CNNs (e.g. ResNet)?
A
Averaging each feature map across its entire spatial dimensions (H x W) into a single scalar, drastically reducing parameter count and preventing overfitting
✓ Correct
B
Averaging pixel values across all images in a dataset
C
Computing the average learning rate across all epochs
D
Normalizing weights to sum to 1
💡 Step-by-Step Explanation & Concept Rationale
GAP converts an (N, C, H, W) tensor directly into (N, C, 1, 1), eliminating millions of parameters associated with flattening into large FC layers.
Q. 20 Artificial Intelligence
Difficulty: Hard (1 Mark)
What breakthrough architectural mechanism did 'ResNet' (Residual Networks) introduce to successfully train ultra-deep networks (100+ layers)?
A
Skip (Residual / Shortcut) connections that perform identity mapping: H(x) = F(x) + x, allowing gradients to flow directly backward without attenuation
✓ Correct
B
Replacing all convolutions with dense matrix multiplications
C
Using genetic algorithms instead of gradient descent
D
Eliminating activation functions completely
💡 Step-by-Step Explanation & Concept Rationale
Residual connections let layers learn perturbation residuals F(x) around identity mappings, overcoming vanishing gradients in 152-layer networks.
Q. 21 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is a 'Bottleneck Block' in ResNet-50 and deeper ResNet variants?
A
A 3-layer residual block using 1x1 conv (channel reduction), 3x3 conv (spatial processing), and 1x1 conv (channel restoration) to reduce computational FLOPs
✓ Correct
B
A narrow bottleneck in computer RAM data transfer
C
A queue of images waiting for GPU processing
D
A layer that halts gradient backpropagation
💡 Step-by-Step Explanation & Concept Rationale
Bottleneck design cuts computation by compressing channel dimensions before expensive 3x3 convolutions.
Q. 22 Artificial Intelligence
Difficulty: Hard (1 Mark)
In 'Inception' (GoogLeNet) architectures, what is the role of 1x1 convolutions?
A
Dimensionality reduction (channel pooling) before expensive 3x3 and 5x5 spatial convolutions, drastically reducing computational FLOPs
✓ Correct
B
Upsampling image resolution by 2x
C
Converting RGB images to grayscale
D
Rotating images by 90 degrees
💡 Step-by-Step Explanation & Concept Rationale
1x1 convolutions perform linear combinations across feature channels, shrinking depth while adding non-linear expressiveness.
Q. 23 Artificial Intelligence
Difficulty: Hard (1 Mark)
What compound scaling method does 'EfficientNet' use to scale network depth, width, and image resolution simultaneously?
A
Compound Coefficient scaling that balances depth (d = alpha^phi), width (w = beta^phi), and resolution (r = gamma^phi) under fixed resource constraints
✓ Correct
B
Arbitrarily doubling network depth until memory runs out
C
Increasing image resolution while keeping depth constant
D
Scaling only the batch size exponentially
💡 Step-by-Step Explanation & Concept Rationale
Tan & Le proved that coordinating depth, width, and resolution scaling yields state-of-the-art accuracy with order-of-magnitude fewer parameters.
Q. 24 Artificial Intelligence
Difficulty: Hard (1 Mark)
What is 'Depthwise Separable Convolution' (used in MobileNet and Xception)?
A
Splitting standard convolution into a Depthwise Convolution (applying 1 spatial filter per input channel) followed by a Pointwise Convolution (1x1 conv combining channels)
✓ Correct
B
Applying convolutions only along the diagonal of an image
C
Convolving across 3D video frames simultaneously
D
Separating red, green, and blue color channels permanently
💡 Step-by-Step Explanation & Concept Rationale
Depthwise separable convolutions reduce computation by ~8x to 9x with minimal loss in accuracy, enabling real-time edge AI on mobile devices.
Q. 25 Artificial Intelligence
Difficulty: Medium (1 Mark)
What is the primary limitation of standard Recurrent Neural Networks (vanilla RNNs) on long sequential data?
A
Exploding and vanishing gradients across long unrolled time steps, preventing the network from capturing long-term temporal dependencies
✓ Correct
B
Inability to process text data
C
Requirement for 3D tensor inputs
D
Lack of matrix multiplication operations
💡 Step-by-Step Explanation & Concept Rationale
Repeated multiplication of transition matrix W_hh across hundreds of time steps drives gradients to zero or infinity, creating short-term memory limits.
Study Stream Progress: Showing 25 of 82 Questions (30%)
Jump to:

Ready to Test Your Retention & Speed?

Now that you have reviewed the study questions and rationales, test yourself in our interactive 1-by-1 practice engine or take the full official timed mock exam.