THE PENN-LEHMAN AUTOMATED TRADING PROJECT: 2003-04

NEWS AND NOTES FOR THE PENN-LEHMAN AUTOMATED TRADING PROJECT: 2003-04

Prof. Michael Kearns
mkearns@cis.upenn.edu

News and Notes 5/14/04: RETRACTION AND REVISION OF BLUE POOL RESULTS

If you thought the Blue pool results were too good to be true, you were right. On doing some analysis of the results, we discovered a scripting error in our (overly) automated process for running the simulations that caused all days from 4/27 forward to be run on that day's data! Hence the steady, repeated stream of profits in the Blue pool from that day forward. Very embarassing, though somewhat reminiscent of the dot-com bubble. The Red pool was run correctly, and there are no revisions to the results there.

We therefore must retract the previously announced results for the Blue pool, and provide the final and official results here. The Blue pool winner, and by a wide margin, is agent ramamoorthy. Overall, profitability in the Blue pool is considerably more sober now.

Profuse apologies to all for this snafu. Since all of the Blue pool winners, both the retracted and the newly crowned, are from the powerhouse UT Austin, I will kindly ask my esteemed colleague Prof. Peter Stone to oversee the complex process of having the winnings transferred between the accounts of the affected parties.

And speaking of Texas and the retraction of results, don't you wish such things were possible in the case of last night's electrifying Spurs-Lakers game?

Again, a good summer to all.

News and Notes 5/10/04: AND THE WINNERS ARE...

Our competition has completed, and congratulations to Corey Kenyon and Olivia Tzou of the Red pool (agent kenyon), and second-time champion Alex Sherstov from the University of Texas at Austin in the Blue pool, with a whopping Sharpe of 5.7 for the 10-day period. Kudos also to Blue pool runner-up Harish Subramanian , also from UT Austin, who was barely nudged out at 5.4.

Final aggregated results are here. Thanks to everyone for participating, and have a great summer! For those of you who have an interest, I may run some summer league action.

News and Notes 5/6/04: THE HOME STRETCH

Results through Wednesday's trading can be found here. Just two days to go.

News and Notes 5/5/04: CORRECTIONS, RESULTS, CLIENT PLOTS

Many thanks to the ever-sharp German Creamer of Columbia for pointing out that the simulations run through Monday were incorrectly computing the fees and rebates on trades. Seems we inverted the command line arguments, so that there was a fee of 0.002 per share for removing a limit order from the books, and a 0.003 rebate for executed limit orders. Fortunately the correct values can be computed without rerunning the simulations.

Yesterday's simulations were run with the correct arguments, and the results through yesterday are here. For days through Monday, we show both the original incorrect values as well as the corrected ones. The effects are relatively minor, though most clients had their profitability reduced a bit.

In terms of the action itself, in the Red pool agent Kenyon has managed to retake the lead over ggc14 but it's a nail-biter. In the Blue pool, agents sherstov and subramanian have opened up a considerable lead over the pack.

Finally, for those that want more detailed views of how the clients are behaving, on crux we have created directories of the form /home/pxs/April2004/[date], where [date] indicates the day of interest. In each such directory there are postscript plots showing the MSFT share price (Island and PXS), client share position, and client P&L before liquidation penalties. For example, /home/pxs/April2004/0503/ggc14.ps gives the plot for agent ggc14 on Monday's simulation.

News and Notes 5/3/04: MONDAY'S RESULTS

Results through yesterday are here. Red race is tightening up a bit, while the wacky profits continue in the Blue pool. Will the bubble burst?

News and Notes 5/2/04: RESULTS THROUGH WEEK 1

We're at halftime, and results through Friday are posted here. Agent ggc14 continues to impress in the Red pool, and the Blue pool remains fierce. Agent sherstov seems to have perfected the art of small but consistent earnings, giving the highest Sharp overall.

News and Notes 4/30/04: RESULTS THROUGH DAY 4

Results through Thursday are now available here. In the Red pool, agent ggc14 has opened up a bit of a lead on a fourth strong day. Even more impressive is the overall performance of the blue pool, where four agents have yet to experience an unprofitable day.

News and Notes 4/29/04: RESULTS THROUGH DAY 3

Results through the first three days of trading are now available here. The Red pool is so far a two-agent battle between ggc14 and kenyon, while the Blue pool is an all-out brawl. The leader so far is agent sherstov --- who some might recall as the best-performing agent back in December.

News and Notes 4/27/04: OFF AND RUNNING --- DAY 1 RESULTS

Results of the first day of the current competition (on Monday's MSFT activity) are summarized here. We are divided into Red and Blue pools as usual. Agent henis is a VWAP agent running in both pools that is instructed to only buy or sell a large number of shares each day, and thus its Sharpe ratio is irrelevant. Detailed information about each client is provided, as well as an overall summary and Sharpe ratio to date (currently undefined since all standard deviations are vacuously zero).

In the Red pool, agents kenyon and ggc14 are off to nice starts, while agent lkt was hammered by failing to liquidate enough. In the Blue pool, agents ramamoorthy, lo and subramanian also opened nicely.

News and Notes 4/19/04: DETAILS OF UPCOMING COMPETITION

The details of the upcoming PLAT competition will be very similar to those of the December 2003 competition. All of the relevant information is given or repeated in this entry.

First, all participants who are using PLAT as their Senior Design or Independent Study project with Prof. Kearns should consider participation in the competition MANDATORY.

The first step that everyone should take is to indicate their intent to participate by sending email NOW to pxs@gradient.cis.upenn.edu. This email should indicate the names of all members on your team, and a brief description of your strategy. This information will help us in planning for the competition.

The deadline for submitting your client is THIS FRIDAY, APRIL 23 at MIDNIGHT. The directions for submission are:

1. On crux, we have created a world-writable directory in /home/pxs/April2004.

2. In this directory, you should create a subdirectory whose name matches the crux login name of one of your team members. This is the ONLY directory you should create, and the ONLY one in which you should place files.

3. In your subdirectory, you should place your COMPLETE client source code, your makefile, your compiled executable, and ALL FILES needed to properly compile or execute your client.

4. In your subdirectory, you should also place a file named README providing EXACT instructions for the invocation of your client in precisely the way you want it run for the competition. Give the command line exactly --- we will be simply cutting and pasting the command line and all arguments that you give us, so be sure you get it right. In the README file you should clearly indicate whether your client is intended to be a VWAP client or a "normal" client. In the README file you should provide any other information we should know for the proper execution of your client. In your README files, please also list all team member names, and provide a brief description of your client's trading strategy.

5. All of you who are submitting VWAP clients MUST in your README files provide us with instructions to invoke your client in either buy shares or sell shares mode, and also tell us how to specify the volume bought or sold from the command line.

The rules and performance criteria for the competition are described below.

APRIL 2004 PLAT COMPETITION: THE RULES

The competition will take place on MSFT stock for the 10 trading days beginning MONDAY, APRIL 26.

(a) In terms of raw performance, there will be a single criterion for the competition, which is the Sharpe ratio of your client's 10-day profit and loss. More precisely, suppose that on the 10 days, the final profits or losses of your client (see below for how we will compute these) are p1, p2, p3,..., p10, where each value might be positive (profit) or negative (loss). Then your client will be judged by the value

(average of p1, p2,...,p10)/(standard deviation of p1,p2,...,p10)

which is (one form of) the Sharpe ratio.

(b) There are no limits on how many shares your client can buy or sell in a day, but your client MUST LIQUIDATE ITS POSITION by the 4 PM close of the market. More precisely, there will be a monetary penalty applied to your profit or loss each day according to how many shares you fail to liquidate by the close. There is a necessary asymmetry between buying and selling here:

Each share that your client is LONG at the close will be simply valued at 0. In this case, we can view the penalty as the price(s) your client paid to buy these shares during the day.

Each share that your client is SHORT at the close will be valued at TWICE the closing price, and your profit/loss will be docked accordingly. In other words, if you are short N shares and the closing price is p, we will deduct 2*p*N dollars from your client's P&L.

(c) PXS will be run in transaction cost/rebate mode for the competition. More precisely, recall that every time PXS executes a trade, one side of the order must have already been sitting in one of the order books, and the other side of the order must have been the "incoming" order. For each share executed by PXS, the party whose order was already in the books shall receive a REBATE of $0.002, and the party that was the incoming order shall pay a transaction FEE of $0.003. This is exactly the policy used by Island.

Items (b) and (c) above will be applied to your clients raw profit and loss figures in order to compute the values p1,...,p10 in item (a) above.

VWAP trading: As mentioned before, there will be a significant amount of automated VWAP trading in the client pool during each day of the competition, so you might anticipate this in the design of your client. If you are a VWAP client, you will be judged not by the above criteria, but by how closely your agent tracks (or better yet, beats) the VWAP.

News and Notes 4/08/04: PLAT LIVES; COMPETITION APRIL 26

It's been a long hiatus, but the project lives, and we'll hold another end-of-term competition for the 10 MSFT trading days beginning MONDAY APRIL 26; source and executable code will thus be due FRIDAY APRIL 23. Details will be provided shortly, but the guidelines and rules will essentially be the same as for the last competition in December (on which you can see details below).

News and Notes 12/20/03: COMPETITION COMPLETE

Our 2-week competition has now completed, and the final results are summarized here.

In the red pool, no agent finished the competition profitably. The best Sharpe ratio was achieved by agent ryu, who just missed ending in the black due to a tough Friday.

Results in the blue pool were considerably rosier, with four agents finishing with healthy positive Sharpe ratios. The winner was agent sherstov with a Sharpe of 1.88, and impressive profitability on every single day. Sherstov's modest and balanced volumes also demonstrate that small can be beautiful. Agent wu traded at higher volumes but still managed to profit on 9 days, and finished nicely in second place.

The VWAP agents aslam and henis did a nice job of tracking volume in the two pools, though henis consistently traded too little.

Overall, the competition was a great success. Thanks to all participants, and enjoy your holidays!

News and Notes 12/19/03

Competition results through Thursday can be found here. With just one day to go, it appears that in the red pool only agent ryu will finish profitably, while the blue pool has considerably stronger performance. Agent sherstov may well end in the black on every single trading day, a notable accomplishment despite the low volumes.

News and Notes 12/16/03

Competition results through Monday can be found here.

News and Notes 12/14/03: HALFTIME UPDATE

We're at the halfway point of our competition, and the current results summary can be found here. Clients ryu and ggc14 are fighting a tough battle for primacy in the red pool, while client sherstov leads the blue through small but steady earnings.

News and Notes 12/12/03: DAY FOUR RESULTS AVAILABLE

We're all caught up as results through Thursday are summarized here. Postscript client plots are in the standard directories, as are client raw output files. We recommend that people check their client output with the numbers we are posted for consistency.

News and Notes 12/11/03: DAY TWO RESULTS AVAILABLE

A summary of the competition through Tuesday's trading is here, and postscript plots for each client are in the usual directory on crux (see 12/09 News and Notes). We hope to catch up shortly and be posting results the following day.

News and Notes 12/09/03: DAY ONE RESULTS AVAILABLE

The results from the first day (Monday, 12/08) of the competition are now available. A Excel spreadsheet summarizing the results can be found here. Some further comments:

In general, as each day's results become available, I will post the summary spreadsheet on this page, but more detailed plots of each client's behavior can be found in postscript files named by client in the directory /home/pxs/December2003/date on crux, where for instance "date" for the first day would be 1208.

In the end we have a total of 12 clients in the competition. For reasons of diversity and simulation load we have split the 12 into two pools of 6 clients each, which will remain fixed for the duration of the competition. Separate winners of each pool will be declared. You can see details about the two pools in the spreadsheet.

Agents aslam (red pool) and henis (blue pool) are VWAP agents, and thus will be evaluated on separate VWAP criteria. They will be invoked each day alternating between buying and selling shares --- so on 1208 they were invoked in sell mode, for 1209 they will be invoked in buy mode, etc. They will be invoked to sell or buy a total volume of 200,000 shares each day (though from the first day it appears that client henis vastly undershoots its target, having sold only about 25K shares).

Most agents seem to be behaving reasonably at a high level, and in particular, most agents are successfully liquidating by the close and thus avoiding costly liquidation penalties.

While almost nothing can be concluded from just the first day of results, in the red pool, only PLAT veteran Ronggang Yu from Texas finished the day in the black after all fees and penalties were assessed, and in the blue pool, clients ramamoorthy and sherstov were profitable.

News and Notes 12/08/03: COMPETITION BEGINS TODAY

We did some preliminary testing over the weekend and it appears that most of the submitted clients behave reasonably. The competition will be divided into two pools consisting of 5 and 6 clients, respectively; more details will be provided shortly.

In order to minimize load on crux as we run each day's simulation, for the duration of the competition we are requesting that there be NO ACTIVITY on crux for the period 4 PM - 6 AM each day.

News and Notes 12/05/03: INSTRUCTIONS FOR SUBMITTING CLIENTS

Today at 5 PM is the deadline for submission of client source and executable code on crux for the PLAT competition to begin on Monday. This message contains instructions for submission.

1. On crux, we have created a world-writable directory in /home/pxs/December2003.

3. In your subdirectory, you should place your COMPLETE client source code, your makefile, your compiled executable, and ALL FILES needed to properly compile or execute your client.

News and Notes 11/30/03: CLARIFICATION ON VWAP CLIENTS FOR COMPETITION

A number of people have asked for clarification on VWAP agents, and other have suggested that VWAP trading will be a "component" of their strategy. This is fine, but to be clear: either your strategy has VWAP trading as its exclusive goal, in which case you will be evaluated in the manner described below; or your agent will be evaluated according to the criteria described in the 11/21/03 News and Notes entry.

If you declare you client to be a VWAP agent:

Your client should take command-line arguments for whether it will only buy or only sell shares (you MUST support both), and the volume to be traded

Your client should only do the type of trading indicated (buy or sell), and must hit the specified volume EXACTLY each day, or risk disqualification

Your client's performance will be measured by its competitive ratio to the market VWAP; see e.g. this paper on the theory of VWAP trading for definitions.

News and Notes 11/21/03: DETAILS OF UPCOMING COMPETITION
(Updated 11/26/03 to correct bug in penalty for being short shares at the close)

Dear PLAT project participants --- please read this note in its entirety; it includes details for the upcoming PLAT competition.

I. DATES OF THE COMPETITION AND LOGISTICS FOR ENTERING

The competition will be held on the 10 days of trading during the weeks of December 8 and December 15. All trading will be on MSFT, and the trading hours each day will be the standard 9:30 AM to 4 PM.

The deadline for submitting your client will be:

Friday, December 5 at 5 PM

Submission of your client will occur by your placing your source code, compiled executable, makefile for compilation, and any auxiliary files needed to run your client in a directory on crux that we will announce later.

IF YOU INTEND TO PARTICIPATE in the competition, we would like you to send us email indicating this intention by

Wednesday, November26

NOTE: All participants who are senior designs or independent studies should consider participation in the competition as MANDATORY; it will constitute a non-trivial portion of your grade.

II. THE RULES

The information below is consistent with the 10/31/03 News and Notes entry, but provides some further details.

(average of p1, p2,...,p10)/(standard deviation of p1,p2,...,p10)

which is (one form of) the Sharpe ratio.

(b) There are no limits on how many shares your client can buy or sell in a day, but your client must liquidate its position by the 4 PM close of the market. More precisely, there will be a monetary penalty applied to your profit or loss each day according to how many shares you fail to liquidate by the close. There is a necessary asymmetry between buying and selling here:

Each share that your client is LONG at the close will be simply valued at 0. In this case, we can view the penalty as the price(s) your client paid to buy these shares during the day.

Each share that your client is SHORT at the close will be valued at twice the closing price, and your profit/loss will be docked accordingly. In other words, if you are short N shares and the closing price is p, we will deduct 2*p*N dollars from your client's P&L.

Items (b) and (c) above will be applied to your clients raw profit and loss figures in order to compute the values p1,...,p10 in item (a) above.

VWAP trading: As mentioned before, there will be a significant amount of automated VWAP trading in the client pool during each day of the competition, so you might anticipate this in the design of your client. More detail will be provided on this shortly. For those of you building VWAP clients (I know of two such teams, Henis/Zlotnicka and Aslam/Ivanova; if you are also doing VWAP you should contact me ASAP), there will obviously be separate criteria for judging your clients. Details to be determined, but it will essentially amount to how closely you can track the VWAP.

Feel free to email with questions on the above, and Good Luck!

News and Notes 10/31/03: COMPETITION IN EARLY DECEMBER.

This entrycontains a number of remarks regarding the PLAT project, including news about a competition to be held in early December. Some of the comments apply primarily to those of you participating for some form of credit (senior design, independent study, etc.) --- I will mark such comments with a (C).

First of all, I hope all of you have spent some time at least designing and thinking about your strategy, and ideally starting implementation and testing. If you haven't, and are serious about the project, I recommend you begin immediately. This is especially urgent for those in category (C).

On a separate note, several of you have written to me recently requesting meetings to discuss your ideas. I apologize I have been difficult to reach due to my schedule. Next week I am out of town, but after that I have more flexibility and will begin taking meetings with those who are interested. I will probably also try to schedule a group session to answer questions the week after next, most likely early some morning before classes.

In early December (exact dates to be determined), we will hold our first formal trading competition on the project. While the precise rules will be announced later, you can roughly plan for them to be as follows:

The primary performance criterion will be the Sharpe ratio. It is possible that a test for statistical significance for profitability will be considered as well. Thus, unlike past competitions, there will not be a large set of point categories for scoring clients.

Unlike past competitions, there will be no share position limits. Your clients can hold share positions as large as you like. However, see the next item.

Your client must liquidiate its position by the close of each day. More precisely, we will probably implement some kind of monetary penalty for any shares your client owns or owes at the close (4 PM). One possibility is that if you own shares, they will simply be valued at 0, and if you owe shares, you will be charged twice the closing price per share. In any event, this means that you must design your clients with liquidation firmly in mind --- you will need a deliberate strategy for handling this.

The competition will be run using a soon-to-be-released version of PXS that implements transaction costs. These transaction costs will be implemented in a manner identical to those of Island for NASDAQ stocks, which is described here.

There will likely be a significant amount of VWAP trading in the virtual client market each day of the competition. You are encouraged to think of how this might influence your strategy, or how your client could benefit from it.

For those in category (C), performance in the competition will form a non-trivial part of your grade.

News and Notes 9/23/03: INITIAL ASSIGNMENTS; JOB AND LOAD MANAGEMENT; PROF KEARNS OFFICE HOURS NEXT WEEK.

First of all, it seems that most of you have now successfully obtained accounts on crux and completed the simple client install and exercise requested in the 9/17 News and Notes entry. If you have not yet completed these steps and intend to participate in the project, please complete them immediately. Failure to do so may exclude you from the project; we need to have an accurate count of how many serious participants we have for resource planning.

Second, please note that it is your responsibility to manage all of your jobs and processes on crux, and to make sure you do not have any hung or runaway processes. If we see a job that looks like a runaway, we may summarily kill it as root, but everyone's life will be easier, and the load average lower, if you make sure you do not have any runaways. The easiest way of doing this is via the command ps -aux | grep [username] where [username] is your login. This will show you a list of all processes you have running. If you don't recognize a job, or it has a suspiciously long CPU runtime, you should kill it. This can be done via the command kill -9 [pid] where [pid] is the process id number given in the ps command. Note also that runaway jobs with output sent to a file can easily overrun your disk quota.

Finally, I am going to make myself available on the afternoon of next Tuesday, September 30 from 1:30 on to meet individually with people or teams who want to discuss strategy ideas for the project. Please send me email (mkearns@cis.upenn.edu) to set up an appointment.

News and Notes 9/17/03: GETTING STARTED ON CRUX.

I hope that by now accepted participants have obtained an account on crux.cis.upenn.edu and an assignment of port numbers by emailing Elliot Feng at fengyi@gradient.cis.upenn.edu. If not, you should do so immediately.

Once you have your account, I would like all of you to take the first steps towards developing strategies and running simulations on crux. In particular, you should read the Tutorial for Developing and Executing PLAT Clients. Please read this document in detail, as it contains crucial information for getting started.

In particular, I would like ALL PARTICIPANTS to do the following exercise: follow the instructions in the section "PLAT's Hello World", which will result in the generation of two files: out.pxs (the simulator output) and out.mytrader (the client output). Please email to pxs@gradient.cis.upenn.edu the LAST 500 LINES of out.mytrader. After the simulation has completed, you can do this by executing "tail -500 out.mytrader > foo" and emailing the file foo.

Could all of you perform this simple exercise by 5 PM MONDAY Eastern time? This will let us know that people are understanding the tutorial and successfully running simulations. Penn Senior Design and Independent Study students should consider this an assignment, but I'd like everyone to do it please.

News and Notes 9/9/03: INFORMATION ON OBTAINING AN ACCOUNT.

Once you have been formally accepted onto the project (and not before, please), the first step in getting started is to obtain an account on the main linux server of the project, which is crux.cis.upenn.edu. The steps to obtaining an account are:

Send email to pxs@gradient.cis.upenn.edu. In general, this is always the only place you should send email for technical, systems issues. In the email, you should indicate your full name, your program, and year.

Your crux username will be your last name. Once you have received your account information, you should log on to crux and change your initial password.

You will also be assigned a set of 10 consecutive port numbers on which to run your simulations. These are the only port numbers you should ever use.

Your account will have a quota of 400M of disk space. The output files generated by clients and pxs are sufficiently large that you will have to learn to manage your quota somewhat carefully.

News and Notes 9/8/03: NOTE TO POTENTIAL UNDERGRAD PARTICIPANTS.

For those of you who are undergrads and wish to participate in the project this year, please send me a text email with the following information:

Your name, major(s), year, and GPA. If you are a senior design candidate, please indicate so. If not, please indicate if there is another formal category you'd like to participate under, such independent study, or whether you simply want the research experience. In general, I will favor people who can make a yearlong commitment to the project, as opposed to one term.

Your relevant interests and background in computer science, financial markets, economics, and related topics.

If you have any, thoughts on what kinds of strategies or problems you'd like to investigate on the project.

If you'd like to attach a resume, feel free to do so, though it is not required.

As soon as I get the information above, I'll start making decisions about project participation.

News and Notes 9/5/03: PLAT 2003-04 GETTING STARTED.

We are in the process of getting ready to begin the project for the academic year 2003-04. This year there will be more emphasis on the project and simulator as a research platform, and there will hopefully be a fair number of external participants who are doctoral students and faculty members, or professionals in the field of quantiative finance.

NOTE TO PENN UNDERGRADUATES: A number of you have expressed interest in joining the project, in some cases as your Senior Design, and I welcome such participation. Last year we had about 20 Senior Design students. However, in keeping with the goal of increased reseach emphasis on the project, I will be slightly more selective this year, and will favor those with the requisite computer science skills and a demonstrated background and interest in topics such as finance and economics. I will hold an orientation meeting for interested students this coming Monday, September 8 at 8 AM in the main conference room of IRCS. I apologize for the ungodly hour, but it is the only time I can be certain it does not conflict with classes. (IRCS is at 3401 Walnut --- just go into the elevator lobby behind the Starbucks at 34th and Walnut, and take the elevator to the fourth floor. Take a left out of the elevators, then left again through the glass doors of IRCS.) At this meeting, I will give an overview of the project and discuss what is expected on it. Please RSVP to me at mkearns@cis.upenn.edu if you plan on attending this meeting.

I request that all potential project participants, Penn and non-Penn, please read the following paper providing a project overview:

The Penn-Lehman Automated Trading Project. Michael Kearns and Luis Ortiz. Preprint, 2003.

[PS] [Compressed PS] [PDF]

On the logistics side, we are currently in the process of upgrading two linux servers for use on the project. Once that is finished --- hopefully some time next week --- we will begin issuing accounts on the servers and participants can begin strategy development.