Statistical Learning and Feature Engineering
Subject: Machine Learning
Sample Practice Questions
Question 1: Which of the following statements about PCA is TRUE?
- PCA creates principal components that are correlated with each other.
- The first principal component captures the least variance in the data.
- The principal components are linear combinations of the original features.
- PCA requires the data to be labeled.
Explanation: Principal components are linear combinations of the original features. They are designed to be uncorrelated (orthogonal) and capture decreasing amounts of variance.
Question 2: What is Singular Value Decomposition (SVD) used for in machine learning?
- Classification of data.
- Dimensionality reduction, noise reduction, and matrix factorization.
- Clustering data points.
- Association rule mining.
Explanation: SVD is a matrix factorization technique used for dimensionality reduction, noise reduction, data compression, and solving linear systems in machine learning.
Question 3: What is feature selection in machine learning?
- Creating new features from existing ones.
- Reducing the number of input features by selecting a subset of relevant features.
- Scaling features to a similar range.
- Converting categorical features to numerical ones.
Explanation: Feature selection is the process of selecting a subset of the most relevant features from the original feature set, discarding irrelevant or redundant features.
Question 4: Which of the following is a feature ranking method?
- Forward Selection
- Backward Elimination
- Information Gain
- Exhaustive Search
Explanation: Information Gain is a feature ranking method that ranks features based on their individual predictive power, often using metrics like correlation, mutual information, or chi-square.
Question 5: What is the relationship between PCA and SVD?
- They are completely different techniques with no relationship.
- PCA can be performed using SVD on the data matrix.
- SVD is a type of PCA.
- PCA is used only for classification, SVD for regression.
Explanation: PCA can be computed using SVD on the centered data matrix. The singular vectors from SVD correspond to the principal components, and the singular values relate to the variance explained.
Question 6: What is subset selection in feature selection?
- Selecting a random subset of features.
- Selecting the best subset of features based on some criterion.
- Selecting all features from the dataset.
- Selecting features based on their alphabetical order.
Explanation: Subset selection involves searching through the space of possible feature subsets to find the optimal subset that maximizes some evaluation criterion (e.g., accuracy, AIC).
Question 7: What is the difference between feature extraction and feature selection?
- Both are the same thing.
- Feature extraction creates new features; feature selection chooses a subset of existing features.
- Feature selection creates new features; feature extraction chooses existing features.
- Feature extraction is supervised; feature selection is unsupervised.
Explanation: Feature extraction transforms original features into new features (like PCA), while feature selection selects a subset of the original features without transformation.
Question 8: What is the main objective of PCA?
- To increase the number of features.
- To create a new set of uncorrelated features that capture maximum variance.
- To cluster similar data points together.
- To find association rules between items.
Explanation: PCA aims to transform the original features into a new set of orthogonal (uncorrelated) features called principal components, ordered by the amount of variance they capture from the data.
Question 9: What does PCA stand for in machine learning?
- Primary Component Analysis
- Principal Component Analysis
- Predictive Component Algorithm
- Pre-Processing Computational Analysis
Explanation: PCA stands for Principal Component Analysis, a widely used dimensionality reduction and feature extraction technique.
Question 10: Which of the following is a feature extraction technique?
- K-Means Clustering
- Principal Component Analysis (PCA)
- Apriori Algorithm
- Support Vector Machine
Explanation: Principal Component Analysis (PCA) is a classic feature extraction technique that transforms original features into a new set of uncorrelated features called principal components.