Value-at-risk in times of crisis : An analysis in the Brazilian market

The present study aimed at evaluating the predictive ability of the models of market risk estimation in times of financial crises. To this end, models were tested to estimate the financial indicator Value-atRisk (VaR) applied to the daily returns of the BM&FBovespa, the Ibovespa index. Traditional models and those based on the Extreme Value Theory (EVT), considered as two types of distribution, the Generalized Extreme Value (GEV) and generalized Pareto distribution (GPD) were tested. The data relating to two periods of international financial crises termed the 1997 Asian Financial Crisis and the U.S. Subprime Meltdown in 2008 were explored in the study. The results indicated the inefficiency of most statistical models for VaR estimation in moments of high volatility for both periods of crisis. In contrast, the exception refers to the model based on EVT, GPD distribution that proved satisfactory in the estimates in both periods of crisis. The results are in agreement with other studies in the field.


INTRODUCTION
Over recent decades, the "risk" issue has been widely argued in the financial market.The onset of the debate dates back to the 1970s with the growth of the financial industry and the rise of market volatility.The popularization of the capital markets has enabled the exponential growth of turnover on the stock exchange.What was once restricted to financial institutions would then be open to the public domain, increasing the exposure of investments to their inherent risks.
Financial disasters in recent decades, marked by the bankruptcies of large corporations, as well as liquidations of major financial institutions (many of them caused by failure of risk management systems and the awareness on the need for the adoption of regulatory measures by countries), led to the creation of a Committee on Banking Supervision, denoted by the Basel Committee, an advisory body of the Bank for International Settlements (Bank for International Settlements -BIS).
Before the requirements stipulated by regulatory bodies, and in accordance with international agreements, the financial institutions have developed several tools for measurement and control of risk inherent to markets through a statistical and mathematical background.
Authors agree that this article remain permanently open access under the terms of the Creative Commons Attribution License 4.0 International License methodologies for calculating and measuring risk,Value at Risk (VaR), developed in the 90s by the US Bank, JPMorgan, has become a benchmark tool for market risk estimation, as pointed out by Jorion (1997).VaR reflects the maximum loss that an investment may reach in a particular period at a given confidence level.
Given the above, the question arises: In the face of the market risk control requirements, as stated in the regulation of financial institutions (Basel Convention) and, in view of the various estimators of Value at Risk (VaR) in the academic literature, will the existing models be able to provide good estimates of risk at moments of international financial crises?
The present study has as its general goal to evaluate the performance of market risk estimators generated by the application of models based on Value at Risk (VaR), in times of global financial crisis, applied to the Bovespa index, of the BM&FBovespa.
Specifically, this paper seeks to: (a) evaluate the performance of estimation models applied to market risk in periods the financial crises denominated as the 1997 Asian Financial Crisis, and the 2008 U.S. Subprime Meltdown; (b) test the predictive ability of the results obtained from the application of the models based on the Extreme Value Theory as a risk measure.

THEORETICAL REFERENCE
In this section, the theoretical concepts governing market risk are presented, starting with Value-at-Risk (VaR) and its estimation metrics and a review of the major studies upon the topic.

Value-at-risk (VaR)
Market risk occurs by means of the variations in asset prices over time, which creates uncertainty with future values.The higher the fluctuations of returns from an investment, the higher the market risk inherent in such investment.That risk applies to the entire global financial market; it is linked to fluctuations in the prices of stocks, options, bonds, commodities, currencies, sovereign bonds, or other assets traded.
Resolution 3464 of the Central Bank of Brazil defines market risk as the possibility of incurring losses of the resulting fluctuation in market values of positions held by a financial institution.That includes the risks of transactions that are subject to foreign exchange, interest rates, stock prices and the prices of goods (commodities).A more robust measure to control the fluctuations of the values indicated by the regular organs of the financial market is Value-at-Risk (VaR).Such as in Brazil is also referred to as Value at Risk in several specialized publications.
VaR is a measure that estimates the maximum expect-ed loss of an asset at a pre-set period at a given confidence interval (or probability of occurrence).According to Jorion (1997), VaR is a value that represents, for a portfolio or asset, the maximum expected loss arising from market risks for a set period of time and with a probability of occurrence.With its application, the risk manager seeks to predict, with any degree of statistical reliability, the maximum variation in the market value of a portfolio at a given time segment which is generally the period required to effect the settlement of the position.
VaR is a risk measurement method that uses robust statistical techniques heavily used in other fields.Best (1998) simplifies the concept by saying that VaR is the maximum amount of money that can be lost in a portfolio at a given time.
According to Kimura et al. (2009), there are three important features for consideration in the calculation of VaR: 1.
The maximum loss estimate -VaR is a major statistical technique to estimate the maximum potential loss; 2.
The time horizon -refers to the projection period for VaR.VaR is not very suitable for estimating very long time horizons.It is a measure of risk for short-term; 3.
The confidence level -Reflects the desired confidence level for the VaR estimates.
In the light of those details, the Basel Accord established some parameters for analysis and calculation of VaR.The standard set the confidence interval by 99 and 95% for the period of one (1) business day or, for some institutions, one (1) month.The volatility of returns shall be calculated using a robust measurement and the calculation of risk should be assessed via stress measurements.Most risk managers of financial institutions adopt a daily basis for the calculation of VaR, thus determining the maximum loss for one day.The daily VaR is called Daily Earnings at Risk (DEAR).Kimura et al. (2009) point out that although the use of VaR forecasts for one (1) day is quite usual, the characteristics of the market or of the investment portfolio may cause the time horizon to be extended for more days.One can take the example of a hedge investment in illiquid assets, in which managers need to estimate the inherent risk over a period exceeding one day.In this case, a longer-term VaR, assuming five days for the institution to undo the position would be more appropriate to assess the risk of their portfolio.All in all, such increased time horizon would reflect the characteristics of the portfolio more appropriately.
Despite being seemingly simple as a measure, the estimated VaR requires complex approaches, which require specific statistical techniques.The assertive prediction of the market risk shall depend on reliable predictions of the future fluctuations of the asset's returns.Several approaches have been developed over the past decades to estimate the risk.Fittingly, these approaches are based on forecast volatility models.
According to Jorion (1997), the Value-at-Risk assessment requires a comprehensive knowledge of the data distributions.The identification of the features concerning the behavior of the time series considered directs the choice for the best estimation method.
The models dedicated to VaR estimation can be classified in various ways.To Jorion (1997), the models can be divided into two classes: those based on local assessment methodology and the full-evaluation ones.Crouhy et al. (2004) divide the models for VaR estimation into: parametric models based on the parameter estimation of a theoretical distribution and non-parametric models, where the distribution is built through the use of scenarios.The classification that experts and institutions in the financial markets refer to as the most consistent and usual is one that separates the VaR estimation models into two classes: one focuses on the conditional distribution of the process, given the past volatility and another comprises models that focus on the marginal distribution.The former generates the so-called Conditional VaR; the latter includes the Non-Conditional VaR.The most widely used models for estimating VaR Conditional consist of RiskMetrics and those based on ARCH processes.The most widely used models in the estimation of the Non-Conditional VaR include the Empirical simulation, the Delta-Normal, the Delta-Gamma-Normal, the Monte Carlo simulation, and the one based on the Extreme Value Theory.

Studies conducted
The research works on the Value-at-Risk have their onset in the early 1990s, especially those linked to studies that explored the traditional, non-conditional estimating models such as Normal VaR, Historical VaR, the Delta-Normal VaR and extensions.The evolution of the risk estimation techniques based on non-conditional distribution of the process has occurred based on the Extreme Value Theory, hereinafter referred to as EVT.Most of the published studies on the estimation of Value-at-Risk use non-conditional models within the literature to apply the EVT.
McNeil (1998), who is among the pioneers in the use of EVT applied to the field of finance, conducted a study considering the estimation of quantiles in the tail of the marginal distribution of financial return series, using extreme value statistical methods.The researcher proposed a simple method for quantification of extreme values.The distribution used in the study was the GEV (Generalized Extreme Value), adjusted by the Extremal Theta index of the return time series offered by the variations in the share prices of BMW, referring to the period between the years 1973 to 1996.Although it was not a comparative study, the findings evidenced well-adjusted estimation results generated by the application of the GEV model, especially with regard to risk.
Two years later, McNeil and Frey (2000) proposed an alternative method to estimate Value-at-Risk and the risk measurements present in the financial market.The method involved an approach that combined the volatility adjustment by a GARCH process and elements of the Extreme Value Theory.In the study, the estimation of extreme values was carried out over a distribution GPD (Generalized Pareto Distribution) of the time series of returns on stock exchange indexes, the S&P 500 index (USA), the DAX index (Germany), the shares of BMW, and the US dollar for the period 1960-1993.It was assumed the GPD distribution for EVT.Through the backtesting of historical daily return series, it was possible to note that the best estimate of VaR was obtained by the method based on conditional distribution.Ho et al. (2000) conducted a study similar to McNeil (1998).To this end, the GEV modeling for estimating the risks associated with the behavior of stock indices of the stock exchanges in Asian countries was applied.The sample consisted of historical series of returns of the indices of stock markets in Japan, Taiwan, Korea, Indonesia and Malaysia, for the period between 1984 to 1996.The study showed that the risk modeling by EVT proved more accurate in comparison to the estimation obtained by applying traditional techniques.The results also revealed that considering the occurrence of a normal distribution would lead to underestimated values for the VaR.
In the study by Gençay et al. (2003), the performance of EVT in VaR calculation was compared with the results of other well-known modeling techniques, such as GARCH, variance-covariance and the historical simulation method applied to stock markets.The models were classified into two groups.The former group consisted of GARCH (1,1) processes, and GARCH (1,1) with t-student distribution.The Latter group comprised historical simulation, the Var-Cov approach, adaptable to the generalized Pareto distribution (GPD) and non-adaptive GPD models.The data utilized include the returns the principal index of the stock of Istanbul, referring to the period 1987-2001 market.Quantile forecasts of GARCH (1,1) proved to be much more volatile compared to the GPD quantile forecasts.That caused the GPD model to consider a more robust forecasting tool, being more practical for implementation, and displaying a more regular performance for VaR measurements.Silva and Mendes (2003) sought to use the EVT to analyze ten stock market indices in Asia, identifying which type of extreme value and asymptotic distribution best fits into extreme historical market events.The empirical tests indicated that the distributions of returns were not characterized by normality, and that the minimum and maximum of the return series could be modeled in a satisfactory manner within a framework of extreme values.
The sample included returns of stock indices of the markets of China, India, Japan, Indonesia, Korea, Malaysia, Singapore, Philippines, Taiwan and Thailand referring to the period from 1990 to 1999.The results showed that the method of VaR estimation by extreme values would be a more conservative approach to determining capital requirements than traditional methods (SILVA E MENDES, 2003) In a similar investigation to Gençay et al. ( 2003), Gençay and Selçuk (2004) focused on the relative performance of the Value-at-Risk measure applied to the stock markets of nine countries referred to as emerging.The authors used the EVT approach to perform a comparison with the application of traditional models of variance-covariance, historical simulation to generate VaR estimates and provide the tail forecasts of daily returns at the 0.999 percentile.The survey data were the time series of returns of the major indices of stock markets in Brazil, Argentina, Hong Kong, Indonesia, South Korea, Mexico, Singapore, Taiwan and Turkey, referring to the period from 1973 to 2000.The GPD distribution was considered in the estimate of extreme values.The results showed that the EVT-based VaR estimates proved more accurate at higher quantiles.According to estimated parameters of the GPD, some of the moments of the distributions do not exist for some of the indexes.Furthermore, the daily return distributions showed different properties on their left and right tails, suggesting that the risks were not equiprobable in such economies.
Brooks et al. ( 2005) compared different models based on extreme values to determine the VaR three LIFFE's futures contracts, working with prices referring to the period from 1991 to 1997.A semi-parametric approach, where extreme events were modeled using the GPD and normal market conditions captured by the empirical distribution function was proposed.The amounts of VaR to this approach were compared with the non-parametric pattern, with an approach that considers the extremal index, and those calculated based on the initialization from unconditional density, and the initialization from a GARCH (1,1).Results showed that, for a test sample, the approach of the proposed semi-parametric extreme value produced superior results to other methods, but the tail index technique also showed consistent results.
Likewise, Bekiros and Georgoutsos (2005) conducted a broad and interesting research, with a comparative evaluation of the predictive performance of various models for Value-at-Risk (VaR).Notably, two methodlogies relating to EVT deserved distinction, the Peaks Over Threshold (POT) and Blocks Maxima (BM).The study was applied to historical series of returns of US dollars, from the Dow Jones stock index, and from the Cyprus Stock Exchange.The aim was to compare the performance of various estimation techniques in markets with different capitalization and trading practices.The sample included the period from 1997 to 2002.The period from 2001 to 2002 was kept for backtesting.The results reinforced previous studies, according to which, at very high levels of trust, the EVT-based methodology produces the most accurate forecasts of extreme losses.
The following year, in view of the electricity market, Chan and Gray (2006) conducted a study that proposed the use of autoregressive models based on EVT as estimators of VaR in the electricity market.According to the authors, the recent deregulation of the electricity markets has increased the importance of risk management.The VaR assessment within electricity markets is undoubtedly more difficult than in the traditional financial markets.That is because the characteristics of the market result in an unusual distribution of returns.Effects such as seasonality, are present in these markets.EVT was adopted to model the tails of the return distribution explicitly.The study involved historical series of daily returns of the electricity markets from Australia, Scandinavia, Canada, New Zealand and the United States, referring to the period 1999-2004.Compared to the results obtained with the application of parametric models and based on simple historical simulation approaches, the proposed EVT-based model worked well for predicting the VaR.Overall, the results were encouraging in suggesting that the proposal based on EVT model is a useful technique in forecasting VaR in the electricity markets.
Also in the industry of energy, but applied to the oil market, Marimoutou et al. ( 2009) conducted a comparative analysis of models for estimating VaR applying EVT under both unconditional and conditional models for predicting Value-at-Risk.The results from these models were compared with those obtained by means of other known modeling techniques, such as GARCH, historical simulation and Filtered Historical Simulation.The data sample corresponded to the historical series of daily returns of the price of Brent oil and WTI, referring to the period from 1984 to and 2006.The results showed that the conditional EVT and procedures of filtered historical simulation provided a better estimation over conventional methods.In addition, the GARCH model (1.1) also provided good results, comparable to the results of two combined procedures.Finally, the results confirmed the importance of the filtering process for successful standard approaches.Dimitrakopoulos et al. (2010) conducted a study similar to that proposed in this paper.In effect, these authors investigated the quantification of market risk of portfolios of stocks in emerging and developed countries during the crisis and post-crisis periods.The EVT models were compared to historical simulation model and Monte Carlo model, the latter two models referred to as traditional tools.In the sample, the historical returns of the stock indices of the stock exchanges in 16 emerging countries (Argentina, Brazil, Chile, Colombia, Mexico, Venezuela, India, Indonesia, Malaysia, Philippines, South Korea, Thailand, Taiwan, China, Hungary and Turkey) and four developed countries (USA, UK, Germany, Japan) were used referring to the period from 1999 to 2002.
Accordingly, the findings of Dimitrakopoulos et al. (2010) indicated that despite the documented differences between emerging and developed markets, the most successful VaR models were common to both asset classes.Moreover, in the case of stock portfolios in emerging markets, most VaR models produce conservative estimates of risk, in contrast to portfolios of developed markets, where most models underestimate the VaR held.Subsequently, estimating VaR during periods of a financial crisis seems to be a daunting task, especially for emerging markets.VaR estimation models are less affected at times of crisis when applied in developed markets.The performance of parametric models of the VAR estimation improves during the postcrisis period due to the inclusion of extreme events in the estimation sample.
Accordingly, with the evolution of EVT, Cifter ( 2011) presented a study of the extreme value theory based on wavelet to estimate the univariate VaR.Wavelets and EVT were combined to forecast the volatility to estimate a hybrid model.In the first stage, wavelets have been used as a general threshold for the Pareto distribution, and in the second phase, EVT was applied with a waveletbased limit.This new model was used to two emerging stock markets: Istanbul and Budapest.The analysis period was from 1986 to 2010, with daily data.The relative performance of wavelet-based EVT was evaluated against RiskMetrics, ARMA-GARCH, GPD, and conditional GPD models.Empirical results show that the EVT-based model, considering wavelets increased the predictive performance according to the number of violations and tail loss tests.The superior performance of the EVT prediction model based on wavelet was also consistent with the requirements of Basel II and demonstrated that such a model can also be used by financial institutions.
From the most current studies, Jesús et al. (2013) tested the EVT to estimate the risk of the foreign exchange market Dollar / Peso.For the study, a sample of daily returns of exchange rates referring to the period 1970-2007 was used.The models tested were of Historical Simulation, Delta Normal and EVT.The results corroborate the findings of previous studies, pointing the best estimation of VaR by EVT, pursuant to backtesting verification.
The analysis of extremes in financial returns series is oftentimes based on the assumption of observations that are independent and identically distributed (i.i.d.).Nevertheless, stylized facts, such as clustering and serial dependence, usually violate the assumption of independence.This notion has been the main motivation for proposing an approach that is able to overcome those difficulties, considering the time between extreme events as a stochastic process.One of the advantages of the method consists of their ability to capture the short-term behavior of extremes without involving a stochastic volatility or a pre-filtration of data, which certainly shall affect the estimate.With that in mind, Herrera and Schipp (2013) proposed a model, denoted ACD-POT, seeking a better estimate for the VaR.The model was compared with several competing approaches such as CAViaR and the GARCH-EVT model.For the study, the returns for the period 1990 to 2008, the shares of Bayer, DAX index and a portfolio prepared by the authors were used.The backtesting results indicated that the method works properly in risk prediction, thereby providing a more accurate estimate.According to VaR estimates, ACD-POT and the GARCH-EVT methods are the only methods that, more often eradicated the threat of clustering violation.Matos et al. (2014) do a study that analyzed the risk exposure of Brazilian Exchange-Trade Funds (ETFs) compared to Ibovespa and the corresponding BM&FBovespa indexes.Methodology used was risk by Value-at-Risk (VaR) in 5% confiance level, with historical simulation, and Expected Shortfall (ES).The Monte Carlo simulation was used to replicate the verified values.The results showed frequency distributions for the losses, and those lower than the VaR, which characterize the expectation of loss measured by the ES.All ETFs presented losses and exposure to risk greater than those observed for the Ibovespa benchmark and its respective benchmark indexes.

METHODOLOGY Data
The data used include the daily returns of the index of the leading stock exchange in Brazil, BM&FBovespa.Termed as Bovespa, the index exhibits the behavior of companies with greater liquidity on the exchange.The period of analysis was from January 1995 to December 2013.In fact, this period covers the two most recent international financial crises: the 1997 Asian Financial Crisis, and the subprime meltdown, 2008.Data were collected from Economática database and operationalized in R, using the Ismev package.
The calculation of returns was performed by the relationship between the price of a given day in relation to the previous day.The following expression shows the formulation of return calculation.
Where refers to the closing price on day t; refers to the return on day t.
The Bovespa index returns were divided into three samples, described as follows: 1.
The period of the Asian Financial Crisis -The first sample comprises a series of daily returns for the period between July 1997 to June 1998, which was used to adjust the estimate of the VaR model.This period was considered the apex of the crisis.For forecasting and test efficiency models a sample of approximately three months, from July 1998 to September 1998 was used;

2.
The period of the Subprime Meltdown -The second sample was composed by series of daily returns for the period between July 2007 and September 2008, which was used for the adjustment of the estimation of the VaR model.For prediction and efficiency of test models, a sample of approximately three months, between October and December 2008 was used; 3.
The Entire Period -The third sample was composed by series of daily returns for the whole period, between January 1995 and September 2013, which was used for the adjustment of the estimation of the VaR model.For prediction and efficiency of test models, a sample of approximately three months, between October and December 2013 was used.

Data analysis
The data analysis was initially performed using descriptive statistics of the time series.This stage consisted of the calculations of the following values: mean, standard deviation, maximum, minimum, skewness, kurtosis, normality test and stationarity.The idea was to analyze the behavior of the series, by checking for possible volatility clusters and stationary data.The normality test used was that of Bera and Jarque (1981).
The next step was to estimate the Value-at-Risk (VaR) for nine statistical methods, six of which were guided by the Extreme Value Theory (EVT).The risks were estimated In-Sample and Out-of-Sample for quantiles of 5 and 1%.Predictions were made for a onestep-ahead daily basis, in accordance with the proposals of the Basel Agreement and Circular (Internal Regulation) 3464, 2007, issued by the Central Bank of Brazil.
The VaR was estimated by EVT, considering the GEV (Generalized Extreme Value) distribution and GPD (Generalized Pareto Distribution).For each distribution, the extremal index was also considered, as well as an adjustment for larger r.The extremal index  is the ratio Ku blocks, where the maximum exceeded a specific threshold u, and the number of daily returns to u exceeded during the entire period under consideration, Nu.These adjustments by EVT followed the methodology postulated by Mendes (2004).
For GEVdistribution, the VaR was given according to the following expression:

H
is the limiting distribution function to the maximum.
Considering the extremal index , the VaR by the GEV distribution is given by the following expression: The VaR at larger r, with GEV distribution is equal to the GEV based on the expression 02, with the difference in the estimation of the parameters    , , , which was considered the r-largest order statistic, instead of the maximum only.
The VaR adjusted forthe generalized Pareto distribution is estimated by the following expression: The estimate with extremal index  is: The last stage of the method was the final assessment of the estimated risks for both regular periods as to the financial crisis.The test used for this evaluation was proposed by Kupiec (1995), given by the expression: Where: x refers to the number of returns that exceed the calculated VaR; N refers to the total sample;  refers to the established quantile.
According to Kupiec (1995), x follows a distribution ) , ( ~f N Binomial x in which the probability of x in sample N is given by . The test follows a chisquare distribution with 1 (one) degree of freedom.

RESULTS AND DISCUSSION
Figure 1 shows the daily returns of the IBovespa index between the years from 1995 to 2013.As shown in Figure 1, the moments of crisis generate large oscillations in the Ibovespa returns over time, also referred to as volatility clustering; the variations accumulate in precise periods and dissipate in the short term.Accordingly, the oscillations in returns in periods of the Asian Financial Crisis of 1997 and the Subprime Meltdown 2008 are visible on the graph.
The data in Table 1 indicate that all returns sample exhibited a non-normal behavior.The results of the Jarque-Bera Normality Test rejected the normality  hypothesis for the behavior of all indices, at the 1 % level of statistical significance, with p-value statistic below 0.01.All distributions presented some degree of excess kurtosis; given that the calculated values for this statistic are superior to three.The non-normality of the returns of the index brings crucial implications, to the extent that it is an indication that the option pricing models (Black-Scholes), asset pricing (CAPM) and risk management (RiskMetrics) can be flawed when applied to these markets.Such models presuppose the normality of distributions they process.The samples of Ibovespa exhibited a distribution closer to the mesokurtic over the period considered.It means that the returns offered by the variations in this index prices were more scattered (not concentrated in proximity to zero).
Table 2 presents the estimation results of the VaR for the period from the apex of the Asian crisis, between 1997 and 1998 and projected in 1998.According to the data presented in Table 2, one can observe that some VaR estimators had a significant performance as shown by the values of the Kupiec test lower than 3.48.The GPD distribution estimators obtained a better performance since, in both cases, either in the traditional way or corrected by the extremal index , the VaR could be better adjusted.Consequently, the results corroborated the findings of Geyçay et al. (2003), Silva and Mendes (2003), Geyçay andSelçuk (2004), andCifter (2011), which pointed out the EVT-GPD as the finest VaR estimators.
Notwithstanding, in light of an analysis of performance, including all models of Table 2, one can observe that, in most cases, those models failed to acceptably generate the market risk estimation.In fact, the models underestimated the inherent risk at the levels of 5% as well as 1%.Thus, the maximum loss calculated by the models ended up below what the crisis, in reality, occasioned.
Another point to rule refers to the estimation for different confidence levels.The values presented herein demonstrate that the models are better evaluators of market risk for the 1% level than for the 5% level.

Table 1 .
Descriptive statistics of the return series of Ibovespa.

period Period of the Asian crisis Period of the sub prime crisis
presents the estimation results of