Empirical Performance of Black-Scholes and GARCH Option Pricing Models during Turbulent Times: The Indian Evidence

Exchange-traded currency options are a recent innovation in the Indian financial market and their pricing is as yet unexplored. The objective of this research paper is to empirically compare the pricing performance of two well-known option pricing models – the Black-Scholes-Merton Option Pricing Model (BSM) and Duan‟s NGARCH option pricing model – for pricing exchange-traded currency options on the US dollar-Indian rupee during a recent turbulent period. The BSM is known to systematically misprice options on the same underlying asset but with different strike prices and maturities resulting in the phenomenon of the „volatility smile‟. This bias of the BSM results from its assumption of a constant volatility over the option‟s life. The NGARCH option pricing model developed by Duan is an attempt to incorporate time-varying volatility in pricing options. It is a deterministic volatility model which has no closed-form solution and therefore requires numerical techniques for evaluation. In this paper we have compared the pricing performance and examined the pricing bias of both models during a recent period of volatility in the Indian foreign exchange market. Contrary to our expectations the pricing performance of the more sophisticated NGARCH pricing model is inferior to that of the relatively simple BSM model. However orthogonality tests demonstrate that the NGARCH model is free of the strike price and maturity biases associated with the BSM. We conclude that the deterministic BSM does a better job of pricing options than the more advanced time-varying volatility model based on GARCH.


Introduction
The foreign currency derivatives market in India has been till recently mainly an over-the-counter (OTC) market dominated by currency forwards, swaps and options.These derivatives were introduced after the process of financial market reforms was initiated in 1993.Since the Indian foreign exchange market is dominated by banks all currency derivatives were introduced in the OTC market with banks as the intermediaries.In order to further develop the derivatives market in India and expand the existing menu of foreign exchange hedging tools available to Indian residents the joint committee of the Reserve Bank of India (RBI) and Securities and Exchange Board of India (SEBI) permitted the trading of currency futures contracts on recognized Indian stock exchanges from August 2008.Futures contracts on the USDINR pair were introduced initially and the success of these contracts led to the introduction of futures contracts on three other currency pairs.Trading of options on the USDINR was allowed by the regulators from October 29, 2010.Both currency futures and options are currently traded on three exchanges in India viz. the National Stock Exchange (NSE), the Bombay Stock Exchange (BSE) and the Multi-Commodity Exchange (MCX).
The seminal work of Fisher Black and Myron Scholes (1973) and Robert Merton (1973) gave the financial world the first model for pricing contingent claims or options.Although this model was originally intended for pricing European options on non-dividend paying stocks it was later adapted to foreign currency options by Garman and Kohlhagen (1983).The Garman Kohlhagen model is akin to the Black-Scholes-Merton (BSM) model with the underlying asset being the foreign currency exchange rate expressed in terms of domestic currency.This adapted BSM is the most popular model amongst practitioners due to its inherent ease of implementation.Empirical studies however have unearthed several systematic differences between market prices of options and prices derived from the BSM model.These differences are known as "biases" and are related to the strike price and time to maturity of the option and the variance of the underlying asset.The BSM has been found to overprice and underprice options on the same underlying asset but with different strike prices and times to maturity.This has resulted in the well-known phenomenon of the "volatility smile" and "volatility skew".The main reason for the existence of the volatility smile is the restrictive assumption of the BSM that the underlying asset returns have a constant mean and constant variance.Advanced option pricing models have tried to incorporate time-varying volatility in place of the deterministic volatility of the BSM model.One such model was developed by Duan (1995) who used a non-linear GARCH (NGARCH) model to price stock options which allows the incorporation of heteroscedasticity in the asset returns.This paper compares the empirical performance of the BSM and the NGARCH models in pricing options on the USDINR currency pair during a recent volatile period in the Indian foreign exchange market.The main contribution of this paper lies in the fact that to the best of our knowledge this is the first paper to empirically examine the pricing performance of these two models in respect of currency options in India.We begin with the null hypothesis that the NGARCH model which incorporates time-varying volatility should be able to price options better than the constant volatility-based BSM during a highly volatile period.We attempt to identify the better model on the basis of comparison of the pricing error and analysis of the pricing bias of the two models.Our results have practical implications for practitioners and regulators alike.
The rest of the paper is organized as follows.Section 2 reviews the existing literature on option pricing models with time-varying volatility.Section 3 describes the two option pricing models being examined.The data and filter rules are presented and the calibration process explained in Section 4. We present the in-sample and out-of-sample performance, pricing error statistics and orthogonality tests in Section 5. We conclude the paper and explain limitations of the study in Section 6.

Literature Review
Some stylized facts of financial market volatility that have been well-documented in the literature are the "persistence" and "clustering" of volatility and the "leverage effect".Volatility clustering refers to the fact that periods of high volatility tend to occur together and so also periods of low volatility.Although asset returns are not generally autocorrelated realized volatility of asset returns is found to be highly autocorrelated and persistent giving rise to the "long memory" effect in financial markets.The "leverage effect" first observed by Black (1976) refers to the negative correlation between current returns and future volatility.The failure of the BSM model in capturing these stylized facts of financial market volatility is attributable to its strong assumption of a constant volatility.Researchers have broadly taken two approaches in order to correct the assumption of constant volatility in the BSM model.One approach is the continuous-time stochastic volatility models to price options.Stochastic volatility models recognize a second source of volatility apart from the volatility of the returns, i.e the "volatility of volatility".Implementation of stochastic volatility models is cumbersome.Early papers adopting this approach include Scott (1986), Johnson and Shanno (1987), Hull and White (1987) and Melino and Turnbull (1990).Heston (1993) was the first to develop a closed-form solution for pricing options on assets with stochastic volatility.
The other approach in literature is the discrete-time time-varying volatility models using the autoregressive conditional heteroscedastic (ARCH) and Generalized ARCH (GARCH) methods developed by Engle (1982) and Bollerslev (1986) respectively.The GARCH model treats the variance forecast for the subsequent period as a function of three elements: a long-run unconditional variance, a weighted squared innovation of the previous period and a weighted variance of the previous period.Thus the variance forecast for tomorrow is "conditional" on the sum of the long-run unconditional variance, the squared innovation or "shock" of yesterday and yesterday"s conditional variance.The application of GARCH models for option valuation led to the emergence of two basic categories of models in this area: the first is the affine model of Heston and Nandi (2000) which yields a closed-form solution and the second is the non-affine model of Duan (1995) which requires numerical techniques for valuation.Duan and Wei (1999) extended the GARCH model to pricing of foreign currency options.Several papers have compared the empirical performance of GARCH models with that of the BSM model and or stochastic volatility models.Hsieh and Ritchken (2000) compared the performance of the Duan GARCH and Heston-Nandi GARCH models relative to each other and to the BSM model.They found that while both GARCH models could explain the maturity and strike-price bias of the BSM model the out-of-sample performance of the Duan GARCH model was better.Duan and Zhang (2001) examined the performance of the GARCH model and the BSM model using data from the Hang Seng Index options before and after the Asian financial crisis of 1998.They concluded that the pricing performance of the GARCH model was better during the volatile period.Lehar, Schittenkopf and Scheicher (2002) compared the GARCH and Hull-White stochastic volatility model with the benchmark BSM model and found that the GARCH model fared better than the other two models in terms of pricing performance.However the performance of all three models in forecasting the value-at-Risk for an options portfolio was found to be unsatisfactory.Yung and Zhang (2003) compared the empirical performance of the ad hoc Black-Scholes model of Dumas, Fleming and Whaley with that of the EGARCH option pricing model and found that although the pricing performance of the EGARCH model was better this superior performance was small and insignificant.Also the hedging performance of the EGARCH model was found to be worse than that of the simpler ad hoc Black-Scholes model.Bollen and Rasiel (2003) examined the performance of the regime-switching, GARCH and jump-diffusion models and concluded that all these models were superior to the BSM model when fitting option prices.On the contrary Harikumar, Boyrie, and Pak (2004) evaluated the performance of the BSM and GARCH option pricing models for call options on three currencies, viz. the sterling, the Swiss franc and the yen and found that the BSM outperformed the GARCH model.Christoffersen, Jacobs, and Mimouni (2006) compared the empirical performance of affine and non-affine option pricing models.They found that the non-affine GARCH model performed better than the affine Heston-Nandi GARCH model while the latter performed better than the affine Heston stochastic volatility model.Huang, Wang, and Chen (2011) studied the pricing performance of the BSM, Hull-White and GARCH option pricing models for the Taiwan Index Options market and concluded that the GARCH model performed best in that market.Kaminsky (2013) examined the empirical performance of different versions of Duan"s GARCH model and the BSM model for pricing options on Poland"s WIG20 stock index.He concluded that the most accurate pricing models for the period under consideration were the Black model with a volatility term structure and the GARCH model with student"s t distribution.Jou, Wang, and Chiu (2013) examine the performance of heterogenous autoregressive (HAR) models based on realized volatility in the SPX options market during the crisis period of 2007-2008.They find that the HAR models are better than the NGARCH model in predicting out-of-sample option prices probably because the former are based on realized volatility which is closer to the implied volatility of options.Badescu, Kulperger, and Lazar (2008) find that an asymmetric normal mixture GARCH model is better at predicting out-of-sample prices of S&P 500 call options than the standard NGARCH (1,1) option pricing model.Although researchers such as Singh, Ahmed and Pachori (2011) have studied the empirical performance of different models for pricing index options in India to the best of our knowledge there has been no study till date in respect of options on the rupee-dollar currency pair.Our paper aims to fill this gap by comparing the empirical performance of the BSM and the more sophisticated GARCH option pricing model of Duan.

Description of the Models
The models examined are the BSM model and the NGARCH model of Duan (1999).

Black-Scholes-Merton Model (BSM)
The BSM model needs no introduction.According to the BSM the price of an option depends upon five inputs: the option strike price, time to maturity of the option, the price of the underlying stock, the rate of interest and volatility of the underlying stock returns.The model is based upon certain restrictive assumptions such as (i) lognormal distribution of underlying returns with a mean µ and constant volatility σ (ii) constant risk-free rate (iii) unlimited short-selling of securities permissible (iv) no dividends on underlying asset (v) absence of transaction costs and taxes.The basic BSM model was adapted by Garman and Kohlhagen (1983) for pricing European options on currencies as follows: Where and C = call option on the currency,  0 = current exchange rate expressed as number of units of domestic currency for the foreign currency, K = option strike price, r = risk-free interest rate for domestic currency, q = risk-free rate for foreign currency, t = time to maturity of the option and  = volatility of the underlying exchange rate.
The only unobservable parameter in the above model is the volatility of the underlying asset.Several studies have examined the efficacy of alternative measures of volatility to be input into the model.Although early work used historical volatility of the underlying asset as an input to the BSM several researchers such as Latane and Rendleman (1976), Chiras and Manaster (1978), Merville and Pieptea (1989) showed that the underlying volatility implied in the traded prices of options is a better predictor of ex-post volatility than volatility estimated from historical time-series.This measure of volatility is known in the literature as "implied volatility".It is that estimate of volatility which when plugged into the BSM causes the model price to be equal to the observed market price of the option.This modified version of the classical BSM model which uses an option-implied volatility instead of a constant volatility is popular with practitioners.We have used this modified form of the BSM in the present study.

NGARCH Option Pricing Model
Duan (1995) was the first to apply the GARCH model for pricing options.Although there are several variants of the GARCH model we have followed the non-linear asymmetric GARCH (1,1) model adapted by Duan (1999) for pricing foreign currency options.This is because a low-order GARCH model incorporating the leverage effect has been generally found adequate for characterizing time-series (Duan, 1999).Within Duan"s framework the underlying exchange-rate return (  ) is assumed to follow a nonlinear GARCH-M model under the physical probability measure P.
Where  +1 , conditional on information up to the time t+1 is a standard normal random variable,   and   are the one-period, continuously compounded domestic and foreign risk-free interest rates respectively and  +1 denotes the unit risk premium for the exchange rate.
The conditional variance forecast is denoted by  +1 which is assumed to follow a GARCH process defined as: where  0 > 0,  1 ≥ 0,  2 ≥ 0 and  1 (1 +  2 ) +  2 < 1 in order to ensure stationarity of the variance.The parameter θ captures the negative correlation between returns and volatility and thus denotes the leverage effect.
It may be noted that the GARCH model subsumes the BSM model because in the special case when  1 =0 and  2 = 0, the variance process becomes homoscedastic and the GARCH model reduces to the BSM model.
In order to price options within the GARCH framework Duan defined an equilibrium price measure Q and showed that the measure Q satisfied the local risk-neutral valuation relationship (LRNVR) when certain conditions are fulfilled.Under the pricing measure Q, where the variance process is: and =  +1 +  +1 , conditional on information up to the time t is a standard normal random variable under measure Q.The new non-centrality parameter is θ* =   +  and the risk-neutral pricing measure can be determined by four parameters which are  0 ,  1 ,  2 and θ*.Using Eq. 4 the exchange rate at maturity T, (  ), can be computed numerically as: Since there is no closed-form solution for this problem the conditional distribution of   given today"s exchange rate   is determined by running Monte Carlo simulations.A large number of random paths are generated and the exchange rates corresponding to each random path are calculated using eqns.( 4) and ( 5).Under the risk-neutral pricing measure Q the call option price at time t with a strike price of K and maturity of T can be expressed as: In this paper we have used 10,000 simulations of the asset price path to estimate the exchange rate and option payoff at the expiry date.The martingale property fails to hold exactly in simulation because of discretization and sampling errors.We have used the Empirical Martingale Simulation (EMS) technique of Duan and Simonato (1998) in order to achieve variance reduction and ensure that the martingale property is satisfied.The technique is applied as follows for n stock price paths with m sub-intervals     each (Chan & Wong, 2013).
At each point of time  1 …   , we calculate , for j = 1, …n Then the corrected j th stock price at time   is given by After this correction the corrected stock price paths at each point of time satisfy the martingale condition.

Data and Methodology
The data under study comprises daily closing prices of call options on the USDINR currency pair traded on the National Stock Exchange (NSE) from May 1, 2013 to October 31, 2013.As explained earlier this period was selected for study because of the extreme volatility witnessed by the rupee-dollar pair over this period.The options are European-style contracts which are cash-settled on expiry.Every option contract is for a notional value of USD 1000 but the premium is quoted in Indian rupees.These contracts are traded on the Exchange during 9.00 a.m. and 5.00 p.m. from Monday through Friday.

Data Screening Procedure
In line with existing literature we have applied certain rules for filtering the data: (i) option prices not satisfying the lower boundary conditions for call options, ( ≥ max ( 0 −  *  − , 0)) were excluded.(ii) options maturing in less than five trading days were excluded as their prices may be quite erratic and not a reflection of their fair value (iii) strike prices with a daily traded volume of less than ten contracts were excluded because illiquid options may not be priced fairly.After applying these restrictions we were left with a data set of 3156 call option prices.
The filtered data have been analyzed along three different lines: (i)as per option moneyness (ii) as per option maturity and (iii) as per moneyness-maturity.We have measured moneyness as the ratio of spot price to strike price (S/K) and segregated the data in five different moneyness categories: Deep-out-of-the-money (DOTM: S/K < 0.85), out-of-the-money (OTM: S/K >=0.85and < 0.95), at-the-money (ATM: S/K >=0.95 and < 1.05), in-the-money (ITM: S/K >=1.05 and < 1.15) and deep-in-the-money (DITM: S/K >=1.15).We have further segregated the option price data into three categories of maturity: short-term (< 30 days), medium-term (between 30 and 90 days) and long-term (greater than or equal to 90 days).
In order to estimate the initial structural parameters and initial conditional variance for the GARCH model we have used historical data of daily returns of the USDINR for a sliding five-year window starting from May 1, 2008 and ending with September 30, 2013 obtained from the website of the RBI.The yields of treasury bills with the maturity closest to the maturity of the option have been used as the risk-free rate of return for the INR and USD and these have been retrieved from publications of the RBI and the database of the Federal Reserve Bank of Atlanta (FRED) respectively.We have excluded all holidays and taken a year to be equal to 252 trading days.

Methodology and Model Calibration
Both the option pricing models examined in this study depend on unobservable parameters such as the spot volatility in the BSM and the structural parameters  0 ,  1 ,  2 and θ* in the GARCH models.One way of estimating these parameters is by extracting them from the physical asset returns using econometric tools such as maximum likelihood estimation.The BSM was originally used to estimate option prices using an estimate of volatility based on historical observations.However such estimation from historical data reflects only past data which may not be representative of the future.Several papers such as Bakshi, Cao and Chen (1997), Bates (1991), Dumas, Fleming, and Whaley (1998), Heston and Nandi (2000), Hsieh and Ritchken (2000) have followed the approach of minimizing the sum of squared errors between actual and theoretical option prices using a non-linear least squares (NLLS) procedure in order to obtain market-implied volatility and structural parameters.Parameters thus determined have the advantage of being based upon forward-looking information contained in option prices.The procedure followed is to minimize the objective function which is simply the sum of squared errors between the market price of the option and its model price.
where Ω is the vector of parameter values to be determined by NLLS,  , (, ) is the market price of the i th option and  , ( 1 ,  2 …   ) is the model price of the i th option.Calibrating the BSM is relatively simple as only one parameter, i.e. the underlying volatility has to be estimated.We have used implied volatility backed out on the "t" day to price options on day "t+1" using the spot price, strike price, time to maturity and interest rates of day t+1 thus giving us out-of-sample estimates of the option price.
In the case of the GARCH model we have used the underlying exchange rate returns data as well as the options data for calibrating the model as per a two-step procedure.
We started with a five-year rolling window of daily exchange rate returns starting from May 1, 2008 ending with April 30, 2013 and estimated the GARCH parameters  0 ,  1  2 and θ* from this data using maximum likelihood estimation (MLE).These initial parameters were used to estimate the conditional variance on the first day of the month, i.e.May 2, 2013 and then Monte Carlo simulations were employed to simulate the option prices on May 2, 2013 as per the GARCH option pricing model.The second step was to calibrate the original MLE parameters to market prices using the NLLS technique.These new "implied" structural parameters were used as initial parameters to simulate option prices for the subsequent day, i.e.May 3, 2013.Again these parameters were calibrated to market option prices using NLLS.At the end of the month the initial five-year window was rolled forward by deleting the earliest month and adding a new month.The structural parameters on the first day of the new month were again estimated by MLE.This two-step procedure has been followed for the entire sample period.The calibrated "implied" parameters for each day have been used to determine one-day-ahead option prices based on the strike prices, interest rates and spot exchange rates of the next day using Monte Carlo simulation thus yielding out-of-sample estimates of the option price.
We have used Excel VBA code (Chan & Wong, 2013) to perform the Monte Carlo simulations required for estimating the GARCH option model prices.Excel VBA has also been used to perform the minimization by opting for the GRG Nonlinear method and limiting the number of iterations to 1500.No restrictions were placed on the time taken for the iterations.

Empirical Performance and Observations
This section documents the implied volatility pattern and the out-of-sample pricing performance of the models.Table 1 depicts the summary statistics of the daily USDINR returns over the period starting from May 1, 2008 to October 31, 2013.It is evident that the exchange rate returns data are negatively skewed and exhibit a large positive kurtosis.The non-normality of the distribution is confirmed by the Jarque-Bera test statistic which is a high 954.34 with a p-value of zero.The graph of the daily USDINR returns in figure 1 clearly reveals the time-varying nature of the volatility and volatility clustering.An option pricing model based on an underlying GARCH process is therefore expected to perform better than the BSM which assumes normally distributed underlying data.The pricing bias of the BSM model is well-documented in the literature.We have used the BSM to back out the implied volatility from the market prices and analyzed the same within each moneyness-maturity bucket.Table 3 summarizes the descriptive statistics of implied volatilities for different moneyness-maturity buckets.Daily USDINR log returns implied volatilities are seen to be highest for short-term options and tend to decline with increase in the maturity of the option.The standard deviation of implied volatilities is higher for short-term ITM and DITM options but declines for medium-term and long-term options.
In accordance with existing research we have used three yard-sticks for measuring the empirical performance of the two models: (i) in-sample pricing errors (ii) out-of-sample pricing errors and (iii) Orthogonality tests.The following three error metrics have been employed to judge the pricing performance of the two models.
1) Mean Absolute Percentage Error (MAPE).This is a measure of the magnitude of the pricing error.
2) Mean Percentage Error (MPE) . This is a measure of the direction of the pricing error.
3) Mean Squared Error (MSE) . This is a measure of the volatility of the pricing error.

In-Sample Performance
In-sample testing of the models reveals the consistency of the parameters with the option prices.Table 4 depicts the mean and standard deviation for the parameter estimates for both models for each month of the sample period.It is observed that the parameters of both models vary considerably over the sample period as evidenced by their high standard deviations.We have evaluated the in-sample performance of both models by comparing the market option prices with the model prices computed using the estimated parameters of the current day.Specifically, we estimate structural parameters Ω  at time t and use these parameters and other inputs such as the spot price, strike price, interest rates and time to maturity at time t in order to price all options at time t.Table 5 shows the in-sample pricing errors for different categories of moneyness.The MAPE and MPE of both models are the highest for OTM options and progressively decline with increase in the moneyness suggesting that OTM options are the worst priced among all categories.The error statistics of both models are comparable for DITM options.The performance of the GARCH model is better for the OTM and ITM options as evident from the lower MAPE and MPE of the GARCH model.The BS model exhibits a lower absolute percentage error in respect of ATM options.Both models consistently underprice options of all moneyness but the extent of underpricing is lower for the GARCH model.
In-sample performance as per maturity of the options is depicted in Table 6.The pricing errors of the BSM are higher for short-term options and long-term options as compared to medium-term options.The GARCH model shows the highest MAPE for short-term options but the largest MPE for long-term options.While both models underprice options of all maturities, the GARCH model shows a lower underpricing of short-term options than the BSM.The in-sample performance across all categories of moneyness-maturity is shown in Table 7.The table clearly shows that the in-sample performance of the GARCH model is better than the BSM in respect of short-term and medium-term OTM options as evidenced by the lower MAPE of the GARCH model.The BSM prices ATM options better than the GARCH model while the performance of both models is similar for ITM and DITM options.The GARCH model shows a lower underpricing for all categories except for the medium-term and long-term ATM options.To sum up, the GARCH model shows a lower MAPE than the BSM in six out of nine moneyness-maturity buckets and a lower MPE than the BSM in seven out of nine moneyness-maturity buckets.The GARCH model is thus superior to the BSM judged according to in-sample performance.

Out-of-Sample Performance
It is well-recognized that a model with a larger number of structural parameters will lead to a better in-sample fit than another model with fewer parameters.Hence it becomes necessary to evaluate the out-of-sample performance of the models to ascertain whether the improvement in fit is simply because of over-fitting or whether the additional parameters have a true forecasting power.This also enables the evaluation of the stability of the model parameters over time.We have examined the one-day-ahead pricing performance of both models by using the estimated structural parameters of the current day for pricing options on the subsequent day.These out-of-sample forecasts have been compared with traded option prices to estimate the pricing error.
While the pricing performance of both models deteriorates when we shift from in-sample assessment to out-of-sample assessment, the deterioration is found to be sharper in case of the GARCH model.We first examined the pricing error statistics for different categories of moneyness.Table 8 shows that the MAPE and MPE of both models are the largest in respect of OTM options and are seen to decrease as we progress towards DITM options.While the BSM model is seen to underprice options of all levels of moneyness the GARCH model is seen to overprice OTM and ATM options and underprice ITM and DITM options.The degree of under-/over-pricing for both models declines with increase in the moneyness.The GARCH option pricing model fares worse than the BSM with higher error statistics across all categories of moneyness.The pricing errors for both models with respect to the time to maturity are depicted in Table 9.It is evident that the MAPE and MPE for the BSM decline with increasing maturity.The pricing errors for the GARCH model however are higher for short-term and long-term options than for the medium-term options.Again the GARCH model results in higher pricing errors for all maturities than the BSM model.For a better evaluation of the out-of-sample performance we have segregated the option data in nine moneyness-maturity buckets.
Table 10 shows that the BSM model exhibits a lower MAPE in eight out of nine moneyness-maturity buckets.The higher MAPE of the more sophisticated GARCH model is surprising and is contrary to what is observed in advanced financial markets.The BSM model is seen to underprice all categories of options except medium-term and long-term ATM options.The GARCH model on the other hand overprices options of all categories except ITM and DITM options of a short maturity.The BSM model also scores over the GARCH model in terms of having a lower MSE in eight out of nine moneyness-maturity buckets.In order to ascertain whether the outperformance of the BSM model is statistically significant we have conducted the Diebold-Mariano test on the squared pricing errors from both models.This test (Diebold & Mariano, 1995) examines the quality of forecasts from different models.Results of the test are shown in Table 11.The test has been run with the R function "dm.test" taking forecast horizon = 1 and power = 2.The null hypothesis of the DM test is that both models have the same forecast accuracy.We conducted a one-tailed test that the BSM model has greater forecast accuracy than the GARCH model.The results of the DM test show that the p-value is zero.We therefore rejected the null and concluded that the BSM model indeed has greater forecast accuracy.
We have computed the log MAPE ratio in order to compare the models with each other.The ratio was computed as given below: This ratio has been computed for each day in the sample period.A positive value of the ratio means that the BSM yields more accurate estimates than the GARCH model while a negative value implies that the GARCH model outperforms the BSM.We have also computed a "wins ratio" which is simply the proportion of days from the total sample on which the GARCH model outperformed the BSM.Table 12 shows the average DMAPE ratio and the wins ratio for each month of the sample period.The pricing performance of the BSM model is seen to be better than that of the GARCH model in the months of May and June.As the volatility increased, however, this outperformance of the BSM is seen to decline as evident from the declining DMAPE ratio.Table 13 shows these ratios separately for the pre-volatility period (the months of May 2013 and June 2013) and the volatile period (July 2013 to October 2013).The BSM clearly outperformed the GARCH model during the pre-volatile period as evident from the positive average DMAPE ratio and the wins ratio.This outperformance was maintained by the BSM even during the volatile period although the extent of outperformance is lower during the volatile period.
Regression results are documented in Table 14.The standard errors reported are robust and heteroscedasticity and autocorrelation-consistent (HAC) in accordance with Newey and West (1987).There is a high degree of predictability of the pricing error for the BSM as the  2 for the regression is a high 45.62%.The coefficients of the moneyness and maturity terms are negative and statistically significant implying that the pricing error declines with increase in moneyness and maturity.The "smile" effect is obvious from the positive coefficient of the squared-moneyness term which is again statistically significant implying that the pricing errors are higher for deep-in-the-money and deep-out-of-the-money options.Thus the moneyness and maturity biases of the BSM are clearly evident from the regression results.The GARCH model regression however has a much lower  2 of 4.28% implying that there is no predictability of the pricing errors from the main explanatory variables considered here.The t-statistics of the moneyness, squaredmoneyness and time to maturity terms are all close to zero implying that these terms are not statistically significant.We can therefore conclude that there is no moneyness or maturity bias present in the GARCH model.

Conclusion and Limitations of the Study
In this paper we have analyzed the comparative empirical performance of the well-known Black-Scholes-Merton (BSM) model and Duan"s NGARCH option pricing model for pricing options on the USDINR exchange rate during the volatile period of May 2013 to October 2013.The empirical performance of the Duan GARCH option pricing model has not been studied very widely because the model lacks a closed-form solution and implementation is computationally demanding.Also to the best of our knowledge till date there has been no study of this model in the context of Indian currency options.Our paper aims to fill this gap.We conclude that in spite of the added complexities of the NGARCH option pricing model this model is unable to outperform the relatively simpler BSM model in terms of pricing performance.This conclusion is in line with that of Harikumar et al. (2004) who found that the BSM outperforms the GARCH model in respect of currency options.It is however contrary to that of Bollen and Rasiel (2003) who conclude that GARCH models are an improvement over the fixed smile model.Our second finding is that the GARCH model is free of the moneyness and maturity biases exhibited by the BSM model as demonstrated by orthogonality tests.This result in is in line with that of Lehar et al. (2002) and Hsieh and Ritchken (2000).
The limitations of this study are two-fold.First, we have focused on a volatile period in the recent history and hence the sample period is short.The second limitation is that we have not analyzed the hedging performance of the two models.We propose to do the same in a subsequent paper.

Table 2
depicts the descriptive statistics of the call option prices over the sample period.It can be seen that nearly 84 per cent of the observations are short-term options and ATM calls make nearly 82 per cent of the observations.

Table 2 .
Descriptive statistics of the call option price (May 2013-October 2013)

Table 3 .
Descriptive statistics of market implied volatility (May 2013-October 2013) It is evident from the table that implied volatilities vary systematically with respect to the strike prices over the sample period and make a smile pattern when they move from ATM options to OTM or DITM options.OTM, ITM and DITM calls exhibit the highest mean implied volatilities and ATM calls the lowest.Also average

Table 4 .
Parameter estimates over the sample period(May 2013-October 2013)

Table 5 .
In-sample error statistics as per option moneyness

Table 6 .
In-sample error statistics as per option maturity

Table 7 .
In-sample error statistics as per option moneyness-maturity

Table 8 .
Out-of-sample error statistics as per option moneyness

Table 9 .
Out-of-sample error statistics as per option maturity

Table 10 .
Out-of-sample error statistics by moneyness and maturity

Table 12 .
Month-wise DMAPE ratio and wins ratio

Table 13 .
DMAPE and wins ratio during volatile and pre-volatile periods we have conducted orthogonality tests to ascertain to what extent the GARCH model is able to address the pricing biases inherent in the BSM model.According to Madan et al. a pricing model is good if its pricing errors do not demonstrate any pattern and are not predictable.We have performed regression analysis on the out-of-sample absolute percentage errors obtained for each day of the sample for both the models using the Eviews 7 package.The APEs have been regressed on three explanatory variablesthe moneyness of the option, squared moneyness and time to maturity of the option.The form of the regression is:

Table 14 .
Results for regressing out-of-sample errors on option characteristicsNote.Reported standard errors are hetereoscedasticity and autocorrelation consistent (Bartlett kernel, Newey-West fixed bandwidth = 9).