ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever et al.
1.0k
Citations
112
Influential Citations
IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics)
Venue
2004
Year
An approach to the online learning of Takagi-Sugeno (TS) type models is proposed in the paper. It is based on a novel learning algorithm that recursively updates TS model structure and parameters by combining supervised and unsupervised learning. The rule-base and parameters of the TS model continually evolve by adding new rules with more summarization power and by modifying existing rules and parameters. In this way, the rule-base structure is inherited and up-dated when new data become available. By applying this learning concept to the TS model we arrive at a new type adaptive model called the Evolving Takagi-Sugeno model (ETS). The adaptive nature of these evolving TS models in combination with the highly transparent and compact form of fuzzy rules makes them a promising candidate for online modeling and control of complex processes, competitive to neural networks. The approach has been tested on data from an air-conditioning installation serving a real building. The results illustrate the viability and efficiency of the approach. The proposed concept, however, has significantly wider implications in a number of fields, including adaptive nonlinear control, fault detection and diagnostics, performance analysis, forecasting, knowledge extraction, robotics, behavior modeling.
This paper addresses a critical challenge in system identification: the need for models that can adapt in real-time to changing process dynamics. Traditional Takagi-Sugeno fuzzy models are typically identified offline, requiring a fixed structure and batch data. The authors propose a paradigm shift by introducing the Evolving Takagi-Sugeno (ETS) model, which continuously evolves its rule base and parameters as new data streams in. This is particularly significant for complex, non-stationary processes where static models fail, such as building climate control, industrial processes, and robotics.
The paper's importance is underscored by its high citation count (1030), indicating its influence on the field of evolving fuzzy systems. It bridges the gap between fuzzy logic's interpretability and the adaptability of neural networks, offering a transparent yet flexible modeling approach. The concept of evolving rule bases has since inspired numerous extensions and applications, making this a foundational work in online learning and adaptive control.
The key innovations of the paper include:
The paper reports testing on data from an air-conditioning installation serving a real building. The results demonstrate the viability and efficiency of the ETS approach for online identification. However, the abstract does not provide specific quantitative metrics such as error rates or comparison with other methods. The lack of concrete numbers makes it difficult to assess the magnitude of improvement, but the successful application to a real-world system suggests practical utility.
The broader impact of this work is substantial. It introduces a new class of adaptive models that can learn and evolve in real-time, which is crucial for modern AI applications involving streaming data. The ETS model has been applied to adaptive nonlinear control, fault detection, forecasting, and robotics, among others. Its transparent fuzzy rules offer an advantage over black-box neural networks in safety-critical domains where interpretability is essential. This paper laid the groundwork for a rich research area in evolving intelligent systems, influencing subsequent work on evolving neuro-fuzzy networks and online learning algorithms.
Alex Krizhevsky, Ilya Sutskever et al.
Ashish Vaswani, Noam Shazeer et al.
Douglas M. Bates, Martin Mächler et al.
Diederik P. Kingma, Jimmy Ba