A survey on data‐efficient algorithms in big data era

Analysis

Why This Paper Matters

In an era where big data is often assumed to be a prerequisite for successful machine learning, this survey directly addresses a critical bottleneck: the data hungriness of leading algorithms. Many real-world applications—such as healthcare, robotics, and scientific discovery—cannot afford the cost or time required to collect massive labeled datasets. Adadi’s work systematically maps the landscape of data-efficient methods, providing a clear taxonomy that helps practitioners navigate the field. By highlighting how different strategies (e.g., unsupervised learning, data augmentation, transfer learning, and algorithm modification) can complement each other, the paper encourages the development of hybrid approaches that are both robust and sample-efficient.

Technical Contributions

• Four-category taxonomy: The paper organizes data-efficient methods into (i) non-supervised algorithms (e.g., self-supervised, semi-supervised), (ii) data augmentation (e.g., synthetic data generation), (iii) transfer learning (e.g., domain adaptation, fine-tuning), and (iv) algorithm modification (e.g., few-shot learning, meta-learning).
• Interplay analysis: It explicitly discusses how these strategies can be combined—for instance, using transfer learning with data augmentation to further reduce data requirements.
• Research roadmap: The survey outlines open challenges, such as the need for theoretical guarantees on data-efficiency and the development of benchmarks for fair comparison.

Results

As a survey, the paper does not present new experimental results. Instead, it synthesizes findings from over 300 cited works, offering a qualitative comparison of the strengths and weaknesses of each strategy. The main output is a structured framework that researchers can use to position their own work and identify underexplored areas.

Significance

This survey fills an important gap by providing a unified perspective on data-efficiency, a topic that is increasingly critical as AI is deployed in data-scarce environments. It has the potential to guide future research toward more practical and accessible machine learning systems, reducing the barrier to entry for domains that cannot rely on large-scale data collection. The paper’s emphasis on the interplay between strategies may inspire novel hybrid algorithms that achieve state-of-the-art performance with minimal data.