Thursday 28 April 2011

Obituary of Professor Hirotugu Akaike





Obituary: Professor Hirotugu Akaike, 1927‐2009
Father of AIC
Professor Hirotugu Akaike, Honarary Fellow of the RSS (1983): born in Fujinomiya City, Shizuoka Prefecture, Japan on 5th November 1927; BA (Mathematics) in 1952 and D.Sc (Mathematics) in 1961, University of Tokyo; Researcher (1952‐62), Section Head (1962‐73), Founding Divisional Head of the Fifth Division for Prediction, Control and System Analysis (1973‐86), Director General (1986‐94), Emeritus Professor (1994‐2009), Institute of Statistical Mathematics, Tokyo, Japan; Ishikawa Prize, 1972; Okochi Memorial Technology Prize, 1980; Asahi Prize in 1988, Purple Ribbon Medal in 1989, Japanese Statistical Society Prize in 1996, Second Class Order of the Sacred Treasure in 2000 and Kyoto Prize 2006; died of pneumonia in Tokyo on 4th August 2009, aged 81.
Hirotugu Akaike, the youngest of four brothers born, with a weak constitution, to a silkworm farmer at the foot of Mount Fuzi, was brought up in the country‐side, where he received his schooling before moving to the Naval Academy of Japan during World War II. The War ended just in time to spare him of the front. After the War, he went to the Dai‐ichi Koto‐gakko (commonly called the Ichiko, which was the best secondary school of its time in Japan, with many illustrious alumni) and then the University of Tokyo, where he graduated in 1952. In the same year, he joined the Institute of Statistical Mathematics (ISM) in Tokyo, in which he spent his entire professional life till his retirement in 1994. He married Ayako in 1957. She was a most devoted and organized wife. He told me, more than once, that she took care of everything for him, down to his travelling money for foreign trips, so that he could concentrate on his research.
Akaike’s research activities in Statistics can be divided into phases before, during and after an interest in time series. The pre‐time‐series phase was essentially theoretical covering, among others, decision processes, evaluation of probability distributions and numerical optimization in statistics. His most important contribution during this phase is the convergence property of the optimum gradient method, which is the key to modern nonlinear optimization; for this he is held in very high regard in the optimization community.
The time‐series phase was ushered in by his collaborative research (with Professor A. Shimazaki, a silk process control engineer) on a stationary process taking only the values of 0 or 1. This so‐called gap process arose from the control of silk production. Akaike exploited his childhood knowledge of silkworm farming and his previous experience with the modelling of traffic flow, to come up with a dynamic control of silk reeling process far superior to the static control chart based on a simple Poisson process. The experience convinced him that significant research in time series analysis could emerge by getting involved with users in the real world. Since then, he made conscious efforts to collaborate extensively with engineers from diverse areas: motor car, ship, cement, thermal electricity generation, airport, and so on. Initially, his tool kits were almost exclusively frequency‐domain based. This was perhaps not surprising in view of his serious interest in designing audio amplifiers; he even considered opening a radio shop once. His most notable contributions of this period include the modelling of the f ‐2 spectral density in 1960, a phenomenon brought to his attention by some airport (runway) engineers; his model was an early bird in the analysis of long‐memory time series. His other significant contribution was the identification and handling of the possibility of rapid phase‐lag change at some frequencies in cross‐spectral analysis in 1962.
In the later part of the 1960s, Akaike was forced to abandon, though not completely, his almost decade attachment to the frequency‐domain methodology. After strenuous attempts, he finally concluded in 1967 that the spectral domain would be practically powerless for the design of feedback controller. In designing his own audio amplifier with multiple feedbacks, he “experienced much difficulty.” Professionally, he was then busy helping his engineers to design an optimal kiln controller in cement production. This was a multivariate time series project because a rotary kiln involves several variables, and the conventional control system based on differential equations was quite hopeless. Together with T. Nakagawa and others, Akaike decided to adopt a time‐domain approach. They chose the linear multivariate autoregressive (AR) model after accounting for the inherent nonlinearity of the system by a saturation‐type filter in 1969. At this point, Akaike was faced with the inevitable and thorny issue of order determination, for his collaborators kept calling him up for an appropriate order in their AR models. Out of necessity, Akaike developed the Final Prediction Error (FPE) method by focusing (quite naturally in this context) his model building on prediction. It should be mentioned, however, that he retained some lingering liaison with the frequency‐domain: he introduced the relative spectral power diagnostic, which is a powerful tool still to be appreciated outside Japan.
On 16th March 1971, as Akaike was taking a seat in the morning commuter train on his way to his office in Tokyo, it suddenly dawned on him that the basic ideas behind the FPE could be used in a context much wider than AR modeling. By replacing a point prediction by a predictive distribution and bringing in the Kullback‐Leibler information in the space of distribution functions, the famous “An Information Criterion” (later more affectionately called the Akaike’s Information Criterion or AIC) was born! It takes the almost Einsteinian form of
AIC = ‐2(maximized log likelihood) + 2(number of free parameters).
Certainly by 1969, he knew before everybody else that the FPE (and hence also AIC) would lead to an inconsistent model selection if there exists a true model of finite order. In 1977, he proposed a Bayesian information criterion (ABIC), with +2 replaced by +O(lnN), that is similar to the BIC proposed by G. Schwarz in 1978 and leads to consistency. (It should be noted that inconsistency of the AIC‐type approach disappears for a non‐parametric AR model.) In 1977 and 1978, he made an excursion into the Bayesian territory. Although not many statisticians realize this, by adopting a Bayesian approach, he had already developed notions which, almost twenty years later, came to be called model uncertainty and model averaging. He also used the ABIC to select the prior distribution in a Bayesian framework. His brand of Bayesian thinking also led to significant advances in seasonal adjustments (via smoothness priors and hyper‐parameters) and the Stein’s problem, the former having points of contact with the currently topical “large P small N problem”. His excursion was, in many ways, not unusual because he always preferred estimation to hypothesis testing, arguing that the only place for the latter is in quality control. However, it is doubtful if he could ever pass the infamous ‘cricket test’ to be accepted as a full Bayesian citizen. In the 1980s, he systematized his AIC approach into what he called the Entropy Maximization Principle based on the entropy developed by L. Boltzmann in statistical mechanics in the 1870s, thus concluding his predictive approach to statistical modelling. On judging by the phenomenal impact of AIC in almost every field of science and technology, there is no doubt that AIC is an intellectual achievement of the highest order. The Kyoto prize in 2006 came as no surprise.
Besides AIC, in the 1970s Akaike obtained a deep result concerning the abstract, control engineering notion of state: in a stochastic system, he identified it as the basis vector of the linear space spanned by the k‐step‐ahead linear least‐square predictors for k ≥ 0. This work was inspired by the visit to his Institute by the control engineer (later also a Kyoto prize winner) Rudolf Kalman in 1971. In 1974, this work culminated in a Markovian representation of stochastic processes, with which he addressed the fundamental issue of identifiability in multivariable autoregressive moving average models as well as block Toeplitz matrix inversion. In all these, his dexterity with control theory, à la Kalman, and time series analysis was in dazzling display. Akaike also turned his attention to bio‐medical applications in the 1970s. As a practical man at heart, he paid particular attention to the development of free software to accompany his theoretical results throughout his research career. The famous TIMSAC package (in its several editions) has played an important role in popularizing the use of time series methods in analyzing dynamic data.
Ayako died suddenly of subarachnoid haemorrhage in 1983, throwing Akaike into deep depression. It was the loving devotion of his three daughters, Yumi, Chie and Maki, that sustained Akaike during this painful period. Later with their encouragement, he re‐married. The tender care of his new wife, Mitsuko, restored him peace and calmness and gradually brought him happiness that lasted till his end. From 1986 to 1994, he spent most of his time and energy running the ISM as its Director General. There were challenges. The most serious one entailed his rallying international support for the ISM to be allowed to continue to exist as a purely research institution. That it remains largely so to‐date, with the additional mandate to train postgraduates, owes much to his international standing and negotiation skills. He was the Founding Head of the first Department of Statistical Science in Japan, within the Graduate University of Advanced Study. He served on the Science Council of Japan and was instrumental in instituting a grant system to enable Japanese statisticians to have, for the first time, direct access to research grants. However, the inevitable effects of ageing and the directorship of a large organization both combined to reduce gradually the intensity of research for the post­sixty‐year‐old Akaike, but sparks of creativity remained visible till almost his last breath.
Retirement marked the beginning of the post‐time‐series or the final phase. Thus, in 1994, he became, as he told me with a touch of humour, “a house‐husband with a working wife”. He also took up golf. As somebody never content to be a mere amateur, he set about uncovering the secrets underlying golf‐swing action. He recounted that he found them when he was forced to lie in bed during a long period of illness; the restriction of movement enabled him to probe the different basic modes of swing with the result of a deeper understanding of the swing action. In 2001 and 2003, he developed a semi‐philosophical approach to the art of modelling: he seemed to claim that information had three sources, namely objective knowledge, empirical findings and observational data. And we should utilize all of them in building a statistical model, bearing in mind the role played by the objective of the modeling.
Akaike was widely honoured. Among the most notable ones are the Purple Ribbon Medal and the Asahi Prize, which are two of the highest honours in Culture and Science in Japan, and the Kyoto Prize, which is one of the highest honours in science in the world. Thanks to his unusually good command of the English language, he was an ideal ambassador for Japanese statistics. He was active on the international scene: Council member of the International Statistical Institute; visiting professor to Stanford, Hawaii, the University of Manchester Institute of Science and Technology (where I first met him in 1973), Harvard, the Chinese Academy of Sciences; co­organisers of US‐Japan Time Series Conferences; Honorary Fellow of the Royal Statistical Society, UK; and others.
Akaike was always very kind to young researchers. Under his guidance, a generation of Japanese time series analysts has emerged from the ISM: Genshiro Kitagawa (currently Director General of the ISM), Makio Ishiguro and others. Speaking for myself, I can still remember the many exciting hours I spent studying his selection of important papers written by others (many of his own papers were still in boxes yet to be opened), all bearing his personal marginal notes in his neat handwriting. The experience was magical: I was given a Narnia wardrobe with which I found a New World. All this was made possible for me, during my six‐month visit in 1974, by sharing half of his Japanese‐size office and having free access to his private library.
At the beginning, Akaike started his youth‐hood by being deeply troubled by the `meaninglessness of life’. He told me that he once contemplated suicide in the 1940s. It was watching the gold‐fish swimming freely in the pond that gave him a ray of hope. In the end, he completed his life’s journey by transforming the ray of hope into the brilliance of a star. He will always be remembered by those whose life trajectories have been fortunate to touch his, however briefly, as a most gentle person of great intellect, integrity and generosity. Now that he has left us forever, the world has lost one of its most innovative statisticians, the Japanese people have lost the finest statistician in their history and many of us a most noble friend.
Howell Tong
5th November 2009 version iii

Professor Peter Whittle's after dinner speech















Tribute to Howell Tong by Professor Peter Whittle FRS 
19th December 2009

It gives us all great pleasure to celebrate Howell’s 65th birthday at this meeting and, more than that, to celebrate his remarkable record of achievement: the steady perseverance and probing insight with which he has led the development of a whole theory for the extension of time series analysis into the nonlinear domain, coping with phenomena which in some cases had scarcely been recognised. We celebrate also his great warmth and generosity, familiar to the many who have become first his followers and then his friends and colleagues, and familiar also to the institutions which have so benefited from his good sense and entirely altruistic initiative.

Howell began his research career about 1972 in control theory, progressing to stochastic control. By about 1976 he had picked up the time series theme, but soon became convinced of the inadequacy of the existing linear theory. As many had, but who felt themselves helpless, partly because the familiar linear theory seemed not to transfer at all, but also because the very richness of possibilities meant that they were lost for a guiding theme. However, Howell’s insight led him to such a theme in the threshold models and their various generalisations whose presentation he and Professor Lim began in their RSS paper of 1980. At the same time, Howell did what he knew he had to do; something at which statisticians, including myself, flinched. He mastered dynamic systems theory, the mathematical theory which seeks to discern a conceptual pattern in the nonlinear models suggested by physics and technology.

However, this is a deterministic theory, and on to it one has to graft the randomness of a stochastic model and the statistics of inference. The purely stochastic aspects of such a generalisation had been considered by authors such as Richard Tweedie, and Professor Cline has given an excellent account and extension of such work in his contribution to the Festschrift. Howell had his own insights, recognising, for example,  that the Liapunov functions of a deterministic stability analysis often give the right tool for the study of ergodicity in the stochastic case. However, over and above this, he tackled the daunting inference question.

Characteristically, he produced the complete package: nonlinear stochastic models in full rigour, practicable statistical analyses, well-posed computer simulations and analyses of real data. However, what is also revealed is his intuition and instinct for the essential, which guides him so well on the path to conceptualisation.

For example, after model-fitting he homes on to the question of determination of dimension, as an essential characteristic. One would imagine that this is a difficult plum to pull out of a complicated pudding, but his instinct led him to a cross-validation approach which reveals a remarkable simplification, reassuringly confirming the instinct which had led Howell to this point.

One of the reasons for considering nonlinear models was to find an explanation for the robust limit cycles which are so often observed. These generalise the notion of a stable static equilibrium, and are associated with existence of an attractor in state space. However, even a limit cycle corresponds to a rather special case of such an attractor, and it is now realised that one does not have to look very far before one finds attractors whose equilibrium has the character now termed chaotic. A chaotic solution looks random, and a principal question has been: can one distinguish such a solution from the random sequences of probability theory?

Howell immediately saw this as an inference question. Suppose one had a model which, in its deterministic form, generates a chaotic sequence. Suppose that one stochasticises the model by introducing process noise into the dynamics; would it then be possible to distinguish the chaotic and the random effects in the time series generated by such a model? The natural reaction is to dismiss this as an impossibly delicate task. On the other hand, there could be quite gross effects; the sensitivity of the whole course of a chaotic variable to initial values presumably implies a similar sensitivity to process noise. Howell was of course quite undaunted; he opened up the problem and, jointly with Professor Chan, found a successful treatment. This required a strong generalisation of existing concepts, e.g by generalisation of the deterministic Liapunov indices and by insistence on  identification of the embedding dimension, the dimension of the space within which the attractor lies.

I mentioned Professor Chan. In the course of his work Howell has attracted many students, who went on to become co-workers and co-authors, making their own independent contributions in the field which Howell had opened. Several are here. These would certainly acknowledge the great stimulus that Howell has been to them, but he has the lightest of touches in such matters. He is unfailingly kind, generous, good-tempered and with a very appealing sense of humour.

Professor Cutler refers to just these qualities when she writes in her contribution to the Festschrift that “Howell brings not only incredible talent and creativity to his work, but generosity, enthusiasm and an unmistakable uniqueness of style”. She also very appositely quotes Pascal: “When we see a natural style we are quite surprised and delighted, for we expected to see an author and we find a man”.

Howell’s quotations from Lao Tze in his 1990 book prompt one to ask whether there is anything distinctly Chinese in his work. I don’t regard this as a delicate question, but as a very proper and interesting one, whose answer is in fact positive. Howell completes, over several years, the construction of a large and complex theory, with all its technical detail and all its real-world latency. He seeks clarification, particularly of issues which, although simply expressed, are the subtlest and of the essence. His style is such as to imbue the whole work with a kind of imaginative vividness and even, at times, an impish humour. I believe that in this he maintains, with great distinction, a long and rich tradition from which we all plainly continue to benefit.

May I propose a hearty toast to this most remarkable and most human of colleagues: Howell Tong.


https://apps.saas.hku.hk/lx2/Conference/NTS2009/album/Dec18/Dinner/IMG_8581.JPG