First steps in text mining with R

Everyone is preparing for Christmas Eve's Dinner.  No one is calling, little email. Looks like a perfect time to start researching text mining in R :)

The problem I'm trying to solve:

• extract keywords from multiple texts
• try to summarize texts > sentence extraction
• group and relate products based on their descriptions > classification / clustering
• add relevant information to text based on similar / related text

I've started with tm package.

• http://text-analysis.googlecode.com/files/Text_Mining_Infrastructure_in_R.pdf
• http://cran.r-project.org/web/packages/tm/vignettes/tm.pdf - introduction to tm package
• http://cran.r-project.org/doc/Rnews/Rnews_2008-2.pdf - introduction to text mining in R
• http://epub.wu.ac.at/1923/1/document.pdf - text mining in R and its applications

Then I've jumped to TextRank algorithm for keywords & sentence extraction. Seems, TextRank is not present in tm, but there is Java source code available so should be possible to call it from R.

Will need to compare TextRank to KEA. The later is implemented for R in RKEA.

Looks promising.

Extracting content from web pages for text mining

For the last three days I was looking for ways and tools to extract content (text) and data (for example emails, but that's just a beginning) from web pages.

The best tool I've found so far is Web Harvest: http://web-harvest.sourceforge.net/

Definitely it's not easy too use (it took me nearly a day to understand how exactly did it work), but looks the most powerful among the tools I've evaluated.

I've started with modifying the example of the simple site crawler, to make it more flexible and make it collect all emails found on a specified web site.

Source of my version of site crawler/email collector is available here: http://www.mediafire.com/?c5cjndslu90c8eo

Then, I've written a simple configuration file to collect email addresses from first 100 links returned by Google search for some specified key phrase.

Source code is available here: http://www.mediafire.com/?jkwu6bbs6ef7ohd

Now, I'm moving to the topic I've had in my mind, when I started working on this project - i.e. extracting text content from web pages and analyzing it.

First step: researching topic: Michael J. Giarlo "A Comparative Analysis of Keyword Extraction Techniques"

Real test of USDJPY intervention may come on Tuesday, after FOMC decision

On September 15th, BoJ intervened on the forex. USDJPY jumped 3,3%, to the level from the end of August. Can this action stop the strenghtening of Japanese yen?

Source: Stooq

An interesting study on Japanese forex interventions in 1990s was prepared by Takatoshi Ito from Institute of Economic Research, Hitotsubashi University.

The amount of money used by the Bank of Japan for the intervention should be disclosed on November 8th on the page dedicated to Foreign Exchange Intervention Operations maintained by the Ministry of Finance in Japan: http://www.mof.go.jp/english/e1c021.htm
The amount of the intervention is estimated at 1 trillion JPY (11,7 billion USD).

The first test of the effectiveness of the intervention may come on Tuesday, September 21st, when FOMC announces its interest rate decision and possibly further plans for quantitative easing.

Finanace and distributed computing does not mix?

Browsing through the list of distributing computing projects on Wikipedia, I've found just two financial projects - MoneyBee and GStock.

The first of them - MoneyBee - was conceived as a way to forecast stock markets using neural networks. It was launched in September 2000. According to the information on the MoneyBee's website, the project failed and is suspended.

Fig. MoneyBee Screensaver

Source: Moneybee - example of distributed computing

GStock launched in 2006 and is still operational. It uses technical analysis to generate buy and sell signals for over 4000 stocks. According to information on the project's website, the signals given by the system were correct in 2 out of 3 instances.

Fig. Exemplary GStock chart with generated signals for Google Inc. over 1 year period

Source: http://www.gstock.com/stock-picks/GOOG-Google-Inc.html

Sources:

• list of distributing computing projects on Wikipedia
• information about MoneyBee project on Wikipedia
• MoneyBee on VisWiki
• information about GStock project on Wikipedia
• Moneybee - example of distributed computing

Gold price far from record in EUR and CHF terms - future uncertain

Fig. Gold in USD, EUR and CHF, 1 year, 2010-09-17

Source: Stooq

Depending on the currency in which you price gold, it has reached the new height in absolute terms (USD), is correcting after a strong fall in July (EUR), or stabilizing after it (CHF).

Based on your choice of currency, you may try to predict the future performance of the gold price. And it's not clear at all.

Lyxor ETF WIG20 - the first ETF coming to Warsaw Stock Exchange

Lyxor Asset Management, a division of Societe Generale Group, plans to launch the first Polish ETF (Exchange Traded Fund) in September.

Lyxor ETF WIG20 will track movements of 20 biggest Polish stocks included in the main index of the Warsaw Stock Exchange - WIG20.

Key parameters:

• quoted on the WSE
• price = WIG20 * 1/10
• 0,5% annual management fee
• selling short?

Introduction of this ETF may allow implementation of some arbitrage strategies using either stocks or index futures. Especially if ETF's tracking error will be noticeable, what can happen in times of higher volatility.

see presentation about the ETF: www.lyxoretf.pl/fileadmin/user_upload/ETF/PL/Brochures/LYXOR_ETF_WIG20.pdf

more information: www.lyxoretf.pl

Comparing states of economies of US, EU and China with SENSEnews

I've just started a trial with semantic news service SENSEnews.

As the first test, I've generated charts visualizing six months information flows about the US, EU and Chinese economies.

I've added some additional queries about performance of euro or probability of bubble forming in Chine to the default set of queries predefined in the system.

As an example, I'm showing below results generated by SENSEnews for the query about possible bubble in China:

Is there a bubble in China?

boobdex

$LVS Just because there is a bubble in China, means Asians want to game even more to make back the money they lost

Twitter - Sep, 02 12:03 (EST)

delete

China's Property Market in 'Very Big Bubble,' StarRock Says

Property bubbles exist in some of Chinas large cities, the Peoples Daily reported Aug. 23, citing Ba Shusong, deputy head of the financial institute of the State Councils Development Research Center. He said there is no evidence of a nationwide property bubble in China. China, which hasnt raised interest rates since December 2007, may start increasing borrowing costs from the fourth quarter because of accelerating inflation, large wage increases and loose liquidity conditions, RBC Capital Markets said after the release of July inflation data on Aug. 11. The inflation rate accelerated to 3.3 percent in July, the fastest pace in 21 months, pushed up by floods that destroyed crops. The government is still studying property tax and plans to introduce property tax reform, Xinhua News Agency reported Aug. 24, citing Xu Lin, director of the finance department at the National Development and Reform Commission.

BusinessWeek - Sep, 02 10:01 (EST)

delete

S&P warn that real estate bubble is lurking in China

Asharp pickup in real state prices in China is a key risk to the nation's economic growth and is leading to a classic bubble situation, according to Standard & Poor's. It is forecasting that prices will rise 10.3% this year to an unsustainable level and they will need to come down if they are not to affect economic performance. 'The property market looks darn good, it's been going up, and this is a classic bubble. Prices are rising to an unsustainable level,' said David Wyss, chief economist at S&P. The property sector contributes about 20% to the Chinese economy through real estate investments and related industries, according to Citic Bank International. Since April, China has restricted pre-sales by developers and curbed loans for third home purchases to try to cool the market after excess liquidity stemming from record credit growth last year buoyed demand. But property prices are showing signs of slowing. They have gone up 10.3% from a year earlier in July, the slowest pace in six months.

Arendator.ru - Sep, 02 08:16 (EST)

delete

7Steps2Wealth

China's Property Market in 'Very Big Bubble,' StarRock Says: 2 (Bloomberg) -- China's property market is in a "ver... http://bit.ly/9e2V3Z

Twitter - Sep, 02 04:42 (EST)

delete

chooze2move

S&P warn that real estate bubble is lurking in China | Asia | News http://goo.gl/3QAz

Twitter - Sep, 02 00:30 (EST)

delete

S&P warn that real estate bubble is lurking in China

:: Analysts warn that flat property lending outlook in UK could result in real estate double dip :: The price of detached properties in the UK have risen the most in past year, report says :: :: UK property market prices falling as downward trend kicks in, latest figures show :: Cuba approves 99 year leases for foreign property investors :: Singapore govt introduces new measures to cool real estate market :: Regulations for all new built property in Wales could add up to 10% to costs, consultants warn :: US real estate foreclosures fall marginally but mortgage delinquencies increase to bring more gloom

Property Wire - Sep, 01 13:14 (EST)

delete

10 Things You Need To Know Before The Opening Bell (AAPL, BKC, SPY, GLD)

Gus Lubin | Sep. 1, 2010, 7:26 AM | 916 | 1 A A A x Email Article From To Email Sent! You have successfully emailed the post. Good morning. Here's what you need to know: Asian markets were mostly positive in overnight trading, with the Nikkei recovering 1.17%. European markets are all higher in early trading, and U.S. futures point to a higher opening. See overnight action in charts > ADP Employment numbers come out at 8:15 AM. Forecasts expect 13K added jobs, after 42K added in July. Follow the action at Money Game > ISM Manufacturing data comes out at 10 AM, with analysts expecting a decline to 52.9 from 55.5. Also watch the EIA petroleum data at 10 AM. Recent economic data has been neutral-to-weak. Check out the gritty details behind the Philly Fed number > The surge in gold reached $1,254, approaching record highs . The yen rise also continues as investors fear global deterioration. See how to know if gold is a bubble > China announced plans for new regulations to curb still-soaring home prices.

Business Insider - Sep, 01 12:20 (EST)

delete

10 Things You Need To Know Before The Opening Bell (AAPL, BKC, SPY, GLD)

Gus Lubin | Sep. 1, 2010, 7:26 AM | 847 | 1 A A A x Email Article From To Email Sent! You have successfully emailed the post. Good morning. Here's what you need to know: Asian markets were mostly positive in overnight trading, with the Nikkei recovering 1.17%. European markets are all higher in early trading, and U.S. futures point to a higher opening. See overnight action in charts > ADP Employment numbers come out at 8:15 AM. Forecasts expect 13K added jobs, after 42K added in July. Follow the action at Money Game > ISM Manufacturing data comes out at 10 AM, with analysts expecting a decline to 52.9 from 55.5. Also watch the EIA petroleum data at 10 AM. Recent economic data has been neutral-to-weak. Check out the gritty details behind the Philly Fed number > The surge in gold reached $1,254, approaching record highs . The yen rise also continues as investors fear global deterioration. See how to know if gold is a bubble > China announced plans for new regulations to curb still-soaring home prices.

Clusterstock - Sep, 01 12:17 (EST)

delete

10 Things You Need To Know Before The Opening Bell (AAPL, BKC, SPY, GLD)

Good morning. Here's what you need to know: Asian markets were mostly positive in overnight trading, with the Nikkei recovering 1.17%. European markets are all higher in early trading, and U.S. futures point to a higher opening. See overnight action in charts > ADP Employment numbers come out at 8:15 AM. Forecasts expect 13K added jobs, after 42K added in July. Follow the action at Money Game > ISM Manufacturing data comes out at 10 AM, with analysts expecting a decline to 52.9 from 55.5. Also watch the EIA petroleum data at 10 AM. Recent economic data has been neutral-to-weak. Check out the gritty details behind the Philly Fed number > The surge in gold reached $1,254, approaching record highs . The yen rise also continues as investors fear global deterioration. See how to know if gold is a bubble > China announced plans for new regulations to curb still-soaring home prices. Fear of a property bubble outweighed positive economic data, as the Shanghai Composite whipsawed and ended down 0.60%.

Business Insider - Sep, 01 11:46 (EST)

delete

ceacom

S&P warn that real estate bubble is lurking in China | Asia | News http://bit.ly/anOPvi

Twitter - Sep, 01 07:43 (EST)

delete

Different frequency fft waves for S&P500

I've added a graph showing fft (Fast Fourier Transform) based waves of different number of frequencies:

What Fourier says about S&P500?

Based on the Fourier-based extrapolation of the current trend of S&P500, the stock market will raise or fall :)

I've implemented extrapolation of trend based on Fourier analysis, and - as you can see below - depending on the range of data used for calculations, S&P500 will raise or fall:

Fig 1. Fourier extrapolation of S&P500 based on 100 daily quotations

Fig. 2 Fourier extrapolation of S&P500 based on 400 daily quotations

In both cases, Quinn-Fernandes algorithm for frequency estimation was used and 20 harmonics matched.

I adapted MQL5 (MetaStock) code of Extrapolator indicator written by qpwr. Thanks a lot!

(the code is not yet optimized for R)

Download source code (R).

Best correlation trajectory match for EURUSD

Another test of correlation-based matching of trajectories. This time EURUSD.

Best match with correlation of 0.9295 was found at the beginning of 1980 (red line):

In addition to the best match, two second best correlation matches were added (green lines).

Even though, based on the best correlation match (and one of the others) you might expect pretty significant growth, one of the presented second best matches suggests continued correction.

The next step would be to verify whether such matching has any sense at all :)

Download source code (R) and data.

S&P500 - current trajectory similar to situation from 1959/1960

If one believes in market repeating itself, one could say that the current situation is the most similar to one from the break of 1959/1960:

The correlation between the current and 1959/1960's periods is 0.9292. If market goes the same way as in the past, we should expect a small market increase of around 0,5% over the next 10 days.

The time will tell :)

Simple perfect arbitrage opportunities on forex

Forex is the most liquid of the financial markets. Therefore it is deemed totally deprived from simple arbitrage opportunities.

However, it turns out that one can find - although rarely - perfect arbitrage opportunities even on the most liquid currency crosses.

For example, I've analyzed intraday data (with 10 mins. frequency) for EURJPY, USDJPY and EURUSD for the period of  March 2nd, 2009 - August 20th, 2010. That gives some 54,700 datapoints for each of the crosses.

I defined a perfect arbitrage opportunity for these three crosses, as a situation when difference between EURJPY and USDJPY*EURUSD exceeds 0.1 yen. Then one would be able to sell short EURJPY and at the same time buy USDJPY together with EURUSD (or vice versa) and produce risk free return after adjusting for total spread of 0.1 yen.

In that period I've found 25 instances (out of 54,703 quotations, or 0,0457%) when arbitrage opportunity appeared:

Nevertheless, some of them were substantial - the largest giving 1,31% risk free return!

Visualization of the difference between EURJPY and USDJPY*EURUSD shows, that the arbitrage gap is very rare phenomenon indeed:

The difference between crosses is concentrated in a very narrow range that hardly ever creates arbitrage opportunities. Still, the distribution has very long tails (or at least one of them):

The situation changes a little when you take a look at some less liquid crosses - for example EURPLN vs USDPLN*EURUSD.

The number of arbitrage opportunities increases noticeably (to 6,42% of all analyzed quotations), as well as the rate of potential risk free return (note: total spread set at 0.105 zł):

The difference between base cross (EURPLN) and its equivalent pair (USDPLN*EURUSD) becomes much more volatile:

However the distribution of differences looks much more "normal":

Clearly, there still exist perfect simple arbitrage opportunities on forex. It's possible to exploit them using algorithmic trading. They are rare and often they generate low individual returns. But they are risk free.

Data source: stooq.com

Download source code (R) and data

The coming black boxes inside business organizations

Google has recently made available a set of tools for analyzing data using machine learning techniques called Google Prediction API.

As you can read in the recent article in MIT Technology Review:

For example, a bank or credit-card company wanting to use machine learning to build systems that make decisions based on historical transactions is unlikely to have the specialized staff and necessary infrastructure for what is a computationally intensive approach. [...]

Google's black box (...) contains a whole suite of different algorithms. When data is uploaded, all of the algorithms are automatically applied to find out which works best for a particular job, and the best algorithm is then used to handle any new information submitted. [...]

Google does, however, get (...) information [about effectiveness of various techniques] that it can use to improve its machine-learning algorithms. "We don't look at users' data, but we do see the same metrics on prediction quality that they do, to help us improve the service," says Green. The engineers running Prediction API will know if a particular algorithm is rarely used, or if a new one needs to be added to the mix to better process certain types of data.

Source: Tom Simonite "Google Offers Cloud-Based Learning Engine", Technology Review, 2010-08-20

As machine learning and other pattern-recognition technologies become more available and effective, companies will use them more often.

We can learn some lessons from the investment area, where algorithmic trading has already overtaken human discretion at least in terms of number of transactions executed on the US stock exchanges.It's true that quite often trading algorithms are still quite simple, but they are getting better as Moore's law increases the computing power available for them. It seems inevitable that one day algorithmic trading will dominate the investment industry, and human investors will share the fate of chess master Garry Kasparov defeated by IBM's Deep Blue.

Similarly, proliferation of quantitative methods in business may lead to some interesting results. First, managers will be often and often required to know these new technology-enabled methods. Second, the decision making processes, starting with operational aspects, may slowly morph into hardly penetrable black boxes, processing increasing amounts of information generated and gathered by business organizations.


Originally posted at: blog.inlevel.com 

According to today's breaking news, S&P500 will raise or fall :)

Two consequitive news found at finance.google.com today:

1,200 target reachable for the S&P 500 index

China Post - 16 hours ago

The recent economic reports have shown that the economic recovery is petering out, prompting a few economists to consider the possibility of the economy falling into recession. Robert Shiller ...

Why the S&P 500 Stock Index May Fall Below 1,000

DailyFinance - Aug 15, 2010

The S&P 500 made one sharp move down this year from 1,200 in late April to just about 1,000 in the first week in July. The index may be heading below 1,000 in the next month from the 1,079 where ...

Current index level: 1077.

PQ7: two tubulent years, +107,60%

Fig. PQ7 net results (after transaction fees and taxes) for 2 years between mid July 2008 and mid July 2010

Final result after 2 years: +107,60% (approx. 37% annulized)

It has been two years since I've started a proprietary trading experiment called PQ7.

Its goal was to test some hypotheses regarding financial markets and... survive :)

It so happened that the PQ7 experiment coincided with one of the most volatile periods in the history of modern financial markets, what is reflected in performance of the strategy.

I have learned a lot during this period. I know now what I more of less do not know :)

The experiment will continue for the next 12-18 months with modified - hopefully less volatile - trading strategy.

More updates to come.

Cyclical analysis of gold prices

Some investors believe that it is a good time for buying more gold now, for Q4 is usually positive for gold.

This believe is based, among other, on the assumption jewelery demand increases at the end of the year.

Indeed, if you take a look at the average monthly returns over the last 41 years (1969-2010), price of gold was increasing in September, November and December:

Fig. Average monthly changes

Both Q3 and Q4 were on average positive for gold, and stronger then Q1 and Q2:

Fig. Average daily changes in particular quarters

However if you take a closer look at the distribution of monthly changes, the pictures gets less clearer. September was historically a very volatile month, and while average returns for 3 out of 4 months in Q4 were usually positive, the probability of loss was still substantial:

Fig. Distribution of monthly changes

The randomness of changes is even more visible, when you combine two consecutive months - now average bi-monthly returns are very close to zero.

Fig. Distribution of bi-monthly changes

And still more when you analyze daily data - here you can see that daily changes oscillate around zero each month, without a clear tendency:

Fig. Distribution of daily changes in particular months

You can also take a closer look at the distribution of changes in potentially the most beneficial months, such as September and November - in the former positive average return was generated by infrequent high results, and in the later the distribution is rather symmetrical with some outliers on the negative side:

Fig. Histogram of changes in September

Fig. Histogram of changes in November

Hence, based on the available historic data I cannot agree that the probability of gain from investing in gold now and holding it till the end of the year is significantly higher then making a loss on such an investment.


I just can't see a visible cyclical / seasonal pattern in gold prices. But maybe somebody else can and would correct me?

To see daily chart of gold futures prices go to: http://www.wolframalpha.com/input/?i=gold&a=*C.gold-_*Commodity-

You can also find my bookmarks for articles about gold at: http://delicious.com/reakkt/gold

Reorganizacja BP w ramach przepisów upadłościowych może być mniejszym złem

Flash Player 9 or higher is required to view the chart
Click here to download Flash Player now

View the full BP chart at Wikinvest

Od początku bieżącego roku, akcje BP plc spadły o około 50%. Kapitalizacja spółki obniżyła się do 85 mld USD. Tymczasem szacunki kosztów związanych z wyciekiem ropy w Zatoce Meksykańskiej sięgają 100 mld USD. BP zgodziło się na wpłacenie na rachunek powierniczy 20 mld USD (przy rocznym zysku netto w ostatnich 3 latach w przedziale 16,8-21,7 mld USD), istotnie zmniejszając swoją płynność finansową. Przy niemożliwych do oszacowania ostatecznych kosztach kryzysu, coraz bardziej prawdopodobnym rozwiązaniem wydaje się bankructwo BP, co przynajmniej według amerykańskich przepisów (Chapter 11 - reorganizacja przy ochronie przed wierzycielami), pozwoliłoby na ograniczenie ryzyka firmy i... umożliwiło jej przetrwanie. Wywołałoby to jednak zdarzenie kredytowe, mające znaczenie dla długu firmy oraz instrumentów pochodnych związanych z ryzykiem kredytowym BP.

W połowie czerwca spread na CDS-ach dotyczących rocznego długu skoczył do ponad 1000 bps, czyli 10 punktów procentowych, przy równoczesnej czasowej utracie płynności rynku. 10% spread oznaczał, że ubezpieczenie każdych 10 mln USD długu spółki kosztowało rocznie 1 mln USD. Spready na inne okresy również gwałtownie wzrosły i utrzymują się na alarmującym poziomie (>500 bps). Agencje ratingowe obniżyły rating długu firmy, w przypadku Fitch niemal do poziomu obligacji śmieciowych (S&P: A, Moody's: Aa2, Fitch: BBB). 

Według sprawozdania rocznego BP na koniec 2009 r., długoterminowy dług spółki wynosił 28,7 mld USD. To jednak nie dług spółki może stanowić największe zagrożenie. Znacznie groźniejsze mogą być różnorodne instrumenty pochodne związane z ryzykiem kredytowym firmy.

Jak wynika z noty analitycznej opublikowanej 21. czerwca br. przez agencję Moody's, około 18% wszystkich obserwowanych przez agencję CSO (Collateralized Synthetic Obligations, czyli instrumentów pochodnych opartych na ubezpieczeniach na dług CDS; więcej na temat konstrukcji CDS można przeczytać w ["CDS guide: new structures. CDSs as building blocks", credit ,2003-04"]) zawiera ekspozycje na ryzyko kredytowe BP i spółki związane. 

Pozornie niewielka ekspozycja zawarta w tych CSO (2% dla BP, ok. 3,5% dla wszystkich spółek związanych z obecnym kryzysem naftowym) może paradoksalnie być przyczyną większych kłopotów. 

Po pierwsze - związane z BP dłużne instrumenty pochodne stanowią część ogromnego rynku derywatyw kredytowych (BIS oszacował nominalną wartość CDS-ów na koniec 2009 r. na 32,7 billionów USD). W przypadku wymuszonej zdarzeniem kredytowym likwidacji "zarażonych" ryzykiem BP CSO, może dojść do wyprzedaży części aktywów związanych z pozostałymi firmami zapakowanymi w ten sam instrument. Zazwyczaj dotyka to najpierw najbardziej płynne aktywa. Dodatkowym skutkiem byłby silny skok kosztów długu firm zapakowanych w "skażone" derywatywy.

Po drugie - ukryta w złożonych i nieprzejrzystych instrumentach pochodnych ekspozycja na BP stanowi istotny element niepewności, dotyczący tego, kto i ile będzie musiał zapłacić w przypadku bankructwa BP. I czym sfinansuje swoje zobowiązania?

Kolejnym obszarem ryzyka jest rola BP w handlu instrumentami pochodnymi związanymi z energią, walutami i stopami procentowymi. BP jest prawdopodobnie największym podmiotem zaangażowanym w handel pierwszymi z tych instrumentów. Na koniec 2009 r. bilans spółki wykazywał łącznie ponad 17 mld USD w finansowych instrumentach pochodnych (mniej więcej po połowie w aktywach i pasywach). Bez znajomości struktury pozycji zawartych przez firmę, nie da się jednak oszacować skutków ewentualnego bankructwa firmy dla rynku instrumentów pochodnych. Zabezpieczanie tych pozycji przez przeciwne strony kontraktów zawartych z BP (w tym prawdopodobnie duże banki inwestycyjne) niemal na pewno podniesie zmienność na rynku, szczególnie w zakresie notowań energii. 

Ostatnim elementem, który trzeba uwzględnić w analizie, są różnego rodzaju formalne i nieformalne gwarancje udzielone przez BP swoim kontrahentom i partnerom - np. związane z realizacją zobowiązań umownych. Likwidacja firmy doprowadziłaby do zerwania misternej i rozległej sieci powiązań między firmą i gospodarką. Skutki takiego zdarzenia są nieobliczalne i mogłoby ono dosłownie wstrzymać na pewien czas światową gospodarkę - np. w rezultacie odcięcia klientów BP od dostaw surowców niezbędnych do prowadzenia działalności. Z tego punktu widzenia, bezpieczniejszym rozwiązaniem wydaje się reorganizacja firmy pod ochroną przepisów upadłościowych. A to - o czym była mowa wcześniej - prowadzi do szoku na rynku instrumentów związanych z długiem. Może to jednak być mniejsze zło, a rynek finansowy można przynajmniej czasowo uspokoić kolejnym zastrzykiem płynności.

Źródła:

"BP and Counterparty Risk", Wall Street Pit, 2010-06-26

http://wallstreetpit.com/33059-bp-and-counterparty-risk

"BP Bankruptcy in U.K. Is Obama’s Worst Nightmare: Caroline Baum"

Bloomberg, 2010-06-24

http://www.bloomberg.com/news/2010-06-25/bp-bankruptcy-in-u-k-is-obama-s-worst-nightmare-caroline-baum.html

"BP Default Swaps Rise on Concerns Storm May Disrupt Oil Cleanup", Bloomberg Businessweek, 2010-06-22

http://www.businessweek.com/news/2010-06-22/bp-default-swaps-rise-on-concerns-storm-may-disrupt-oil-cleanup.html

"Death by (collateralised) BP association", FT Alphaville, 2010-06-22

http://ftalphaville.ft.com/blog/2010/06/22/267496/death-by-collateralised-bp-association/

"BP's Bankruptcy Would Impair 117 (18% Of Total) Collateralized Synthetic Obligations, Lead To Pervasive Losses", zerohedge, 2010-06-21

http://www.zerohedge.com/article/bps-bankruptcy-would-impair-117-18-total-collateralized-synthetic-obligations-lead-pervasive

"A Bankrupt BP - Worse For The Financial World Than Lehman Brothers?", Oil Price.com, 2010-06-22 

http://oilprice.com/Energy/Energy-General/A-Bankrupt-BP-Worse-For-The-Financial-World-Than-Lehman-Brothers.html

"Has BP Sprung A Counterparty Risk Leak?", zerohedge, 2010-06-09

http://www.zerohedge.com/article/has-bp-sprung-counterparty-risk-leak

"CDS guide: new structures. CDSs as building blocks", credit ,2003-04

http://db.riskwaters.com/public/showPage.html?page=11387

Link: notowania opcji 2012-01 BP

Thanks to quantitive easing we will get more beautiful money! ;)

Thanks to the quantitative easing there is much more money. And now they're going to be beautiful - the new $100 note will have a cool 3D stripe! ;)

More info: http://www.newmoney.gov/newmoney/default.aspx

Correlation between S&P500 and the US economy

As I've written in my previous post, there was not any straightforward correlation between annual S&P500 returns and GDP nor FFR.

However, when you compare one year's market return to the product of the next year's GDP and FFR (actually 1+FFR), the correlation jumps to 0.61.

GDP*(1+FFR) can be interpreted as the sum of the economic growth and risk free rate in a given time.

Hence we can assume that stock market is predicting future economic situation and future risk free rate. And it's doing it pretty well :)

Nevertheless, due to high volatility, even that market predictions of the future economy state were pretty good, returns produced by the market were only marginally better than the risk free rates as expressed by FFR.

XLS file: http://www.mediafire.com/?mmmyummkzry

Where is the risk premium II?

In the recent 55 years between 1955 and 2009, the average effective Fed Funds Rate was 5,52%. In the same period, the average annual return of S&P500 index reached 6,40% - that's a difference of just 88 pp!

Meanwhile, in the recent 34years between 1976 and 2009, the average yield of Moody's AAA bonds was 8,36% vs 7,63% returned by S&P500. The Fed Funds Rate averaged 6,13%. Even that the risk premium for S&P500 was a little higher - 151 pp - than in the first analyzed period, it was below the premium on AAA bonds!

It is also worth mentioning that no straight correlations between GDP, Fed Funds Rate and GDP existed

correlation      GDP        Fed Funds Rate
S&P500           -0.0383    -0.0689
GDP                         -0.0627

GDP vs Fed Funds Rate vs S&P500 [%] (1955-2009)

XLS file: http://www.mediafire.com/?yonzywyzizj

Data sources: FFR & AAA - Federal Reserve, GDP - BEA, S&P500 - Stooq

Note: the previous post, "Where is the risk premium?" was temporarily removed for revision and update.

Starting playing with signal analysis for modeling returns

I'm starting to test applications of trigonometric functions for modeling returns.

Below a nice graph from Wolfram Alpha:

source: http://www.wolframalpha.com/input/?i=sin(x)%2Bsin(x*x%2Bx),+from+0+to+20

The fall of a country called BP?

Flash Player 9 or higher is required to view the chart
Click here to download Flash Player now

View the full NYSE:BP chart at Wikinvest

From the beginning of the year, and especially from the start the Deepwater Horizon oil spill on April 20th, the market capitalization of BP plc has fallen by some 41,4% to around $106,4 billion. Roughly $75 billion of equity value was destroyed. So far, estimates of total costs to BP reach $30 billion, and some US senators want BP to put $20 billion in an escrow.

Compare this with around EUR 189 billion ($228 billion) of Greek debt outstanding, and $166.8 billion of bank's exposure to Greek debt. Prospects of Greek default sent financial markets tumbling. But the default did not happen, so far. Meanwhile, the BP equity actually evaporated and probability of further loses to investors - both equity holders, creditors ($29 billion long term debt, $24 billion other liabilities) and derrivative counterparts - seems pretty high.

The potential bankrupcy of BP could be similar in size to the default of Greece.

Among the top holders of BP shares are pension and investment funds. These large institutions are usually slow to move. Also, the size of their positions makes the maneuvering difficult. Most probably, they will have to swallow the most of the potential loss. Or rather ultimately the people whose money they manage will have to do so.

This will influence consumer confidence, propensity to save and willingness to invest in equities.

UPDATE 2010-06-14, 10:50 CET

Senior US politicians are pushing President Barack Obama to seek $100bn in damages against BP for the Gulf of Mexico oil spill in an attempt to kill the company.

If such an action were to be taken and won, the Ftse 100 flagbearer would almost certainly collapse into bankruptcy. Many prominent US figures would welcome this as suitable punishment for the environmental devastation off the Louisiana coast. 

source: http://www.independent.co.uk/news/business/news/obama-urged-to-punish-bp-with-100bn-action-1998853.html

Distorted Market Theory 1 - market memory

One of the assumptions of the Efficient Market Theory is lack of autocorrelaction between price changes, which is a result of market not having memory of its history.

However, millions of investors, and more recently "bots" or real-time trading algorithms created by the former, are looking for patterns in market data to profit from them.

In effect, these investors (and their "bots") are the market memory - both short and long-one.

[more]

Market continuity vs jumps

It's not another flash crash yet, but...

UPDATE 2010-06-11, 16:47 CET

source: zero hedge, http://www.zerohedge.com/article/intraday-market-commentary

Results of the pair trading experiment, week 1: Apple vs Google vs Microsoft

So far the proposed pair trading strategy looks promising.

As expected, Apple's stock price has started to decline, while Google's increased significantly and Microsoft's remained at the unchanged level.

It seems, there is still plenty of room for convergence:

link: http://stooq.com/q/?s=aapl.us&c=5m&t=l&a=lg&b=1&r=goog.us+msft.us

Current prices (Friday, June 4th):

• AAPL - $255.96 (-0.46% vs strategy start)
• GOOG - $498.78 (+8.86%)
• MSFT - $25.79 (-0.04%)

UPDATE 2010-06-08, 22:47: dynamic spreadsheet

Before the Tauron IPO

link: http://stooq.com/q/?s=pge&c=5m&t=l&a=lg&b=1&r=ena+cez

The IPO of the Tauron energy group is planned for June 30th. Tauron will join other energy companies currently listed on the Warsaw Stock Exchange: PGE, Enea and CEZ.

Both PGE and Enea seems to be in a down trend from March. Also the whole stock market entered into a correction in May.

Such conditions may influence the results of the Tauron IPO which may be quite a different from the recent PZU's one.

Rozjazd cen Apple, Google i Microsoft - okazja do arbitrażu?

Występująca obecnie, zapoczątkowana w lutym, znaczna różnica w zachowaniu akcji Apple, Google i Microsoft może być okazją do arbitrażu pair trading - zajęcia krótkiej pozycji na akcjach Apple oraz długiej na Google lub/i Microsoft.

Piątkowe kursy tych akcji to:

• AAPL - $257,16
• GOOG - $458,18
• MSFT - $25,80

World Top Investor 2009-10: (plus) +554,56% czy (minus) -83,18%?

Która z liczb robi większe wrażenie: zysk ponad 500% czy obsunięcie kapitału o ponad 80%?

Źródło: http://www.worldtopinvestor.com/pub/index

Tak zadebiutowało PZU

Pozostaje pytanie: co dalej? :)

Zachowanie rozkładów dziennych zmienności logarytmicznych dla WIG20 dla miesięcznych okresów w ostatnich 5 i 10 latach

Dla dodatkowego porównania stabilności rozkładów, przeprowadziłem analizę dla kolejnych 25-sesyjnych okresów (ok. miesiąca) dla danych za około ostatnich 5 i 10 lat (odpowiednio 1260 i 2520 sesji) do 2010-05-07 włącznie.

Najpierw wykres dla ostatnich 5 lat:

Następnie ostatnich 10 lat:

Legenda:

• czarne linie - estymowane rozkłady dla okresów miesięcznych (25 sesji)
• czerwona krzywa - estymowany rozkład dla całego zakresu analizowanych danych w danym przypadku
• zielona krzywa - modelowy rozkład normalny dla średniej i odchylenia standardowego wyliczonego dla całego zakresu analizowanych danych w danym przypadku

Wnioski:

• w obu przypadkach rzeczywiste rozkłady z jednej strony wykazuje silniejsze skupienie wokół średniej, a z drugiej dłuższe/grubsze ogony, niż wynikałoby to z rozkładu normalnego
• rozkłady dla poszczególnych miesięcy znacznie odbiegają zarówno od rozkładu normalnego jak i rozkładu estymowanego dla całego analizowanego okresu