An Empirical Study of Uncaught Exceptions in Open Source Projects

As the title suggests, my current research is an empirical study of uncaught exceptions in Open Source projects. Here is a brief summary of the research.

Summary:

• Use Pex to generate tests on Open Source C# codes
• Take statistics on exceptions being thrown. Report the number of exceptions and type of exceptions that Pex exposes.
• It is expected that the values that cause the exceptions are generally shallow values.
• By shallow values we mean invalid values/parameters.
• The alternative to shallow values are "time bomb" values
• "time bomb" - earlier method call with invalid parameters does not cause immediate exception but a method call later on with valid parameters causes exception due to the earlier method call.
• The "Fix It" feature of Pex may help to determine if the exceptions are caused by shallow values or "time bombs"
• Main procedure

Run Pex on Open Source Project

Classify and group exceptions as either shallow or "time bomb"

Find cases where the "Fix It" feature of Pex can fix whatever is causing the exception to be thrown.

My ability to work has been severely hindered by my preparation for finals. As of now, I've received two grades, both being A's. I'd like to ensure that I maintain these grades for now. I believe I'll be granted more time next semester to work on my Undergrad Research.

The Calculator interface with which I previously began working on turned to be too complicated for my level of experience with C#. Instead of attempting to rewrite the entire program myself, I used the sample version which was provided with the tutorial. I ran the program in the Visual Studio compiler while highlighting key portions of the program to stop at to better interpret the flow of Object Oriented Programming. The code, which consisted of 554 lines of coding became far too lengthy and difficult to comprehend by myself. I'll save the creation of a calculator with a Graphical Interface for my next meeting with Charles. Instead, I began reviewing the abysmally simple tutorials provided here to reinforce the knowledge of C# which was fractured by the overall confusing coding of the "Beginner Calculator". Also for the first time in my coding experience, I did not rely solely on Visual Studio for the construction and compiling of my programs, but made the majority of it in the Command Prompt. Realized I needed to reinstall the .Net Framework in order for the command to compile to activate successfully as well.

Webpages for iTutor

These past few weeks have been very busy as far as school work but I have managed to work on the webpages for iTutor. Basically I am making a webpage for each program that we have been assigned in my CSC 216 class. Now, that we are finished with the semester all the projects have been assigned and I now have all the necessary information to finish these pages. I have completed the webpage for our first project which was a program that read in xml files and compared students musical likings by using a graphing function in java called Jung. I'm currently working on the webpage for our second program which modeled population changes of animals over time. The webpages for the other projects are soon to be finished.

iTutor Formal Writing

In the following 2 weeks, my first priority will be to compile the writing behind the current tool development, with the goal being to encompass all efforts up until this point.

I shall implement this by executing the following steps:
1) Finding publications in a similar area to either cite (i.e., statistics) or compare approaches
2) Write abstract and introduction sections
3) Write related work section
4) Write implementation section

Teaching OOP part 2

The past two weeks has only been involving teaching Khiry OOP. We went over methods, the parts of declaring a method, what are parameters, return values and how objects use methods. We started with simple methods and went into more complicated methods to make sure he understands if he sees alot of code that he can understand what is going on. He established a great understanding of how to create different objects of different types and how to use methods within those objects. We will go over this again when we meet again this week.

Teaching OOP

So the past two weeks I submitted my research paper and taught our new freshman student Khiry Arnold some Object Oriented Programming. We went over the definition of what is an object and what is a class and the differences between them. We did a little exercise to also understand how they work together. I gave a little quiz to see if he can try to figure out how to solve a problem with the tools given. He seems that he has the capability to learn OOP, he just needs time so I am working with him slowly and making sure that each time we meet we go over what we discussed the last time.

iTutor Improvements (continued)

After the last 2 weeks, the remaining iTutor goals are as follows:

1. iTutor needs to check whether it can find equivalent public methods in the student solution, if not, it can flag error.
2. When the methods under test accept non-primitive types, a deep copy of the argument should be made. If not, the method under test of teacher solution may modify the argument before passing it to student solution. Similar techniques need to developed for comparing non-primitive return types.
3. A problem arises where methods can be overloaded, leading to multiple possibilities when instantiating each solution. iTutor shall generate numerous test drivers, each of which focuses on one constructor.
   
4. The logging feature now sends the data to files. The data could be formatted in a fashion where it is easier to comprehend.
   

For the next two weeks, I shall address the above points, as well as improving on other features as required by the addition of the above.

Microsoft Help with Pex

So the past two weeks have been very productive on how Pex works. So I met with Nikolai from Microsoft Pex group and he helped me with my problem that I was having with my complex object. He also told me that I would run into alot of problems because I use a lot of float point arithmetic within my program and Pex has problems with floats. Unfortunately, video games do generally use floating points for movement of objects and placement of objects so this is something that I cannot avoid.

I worked on trying to make Pex and Scurvy.Test work together a little bit more but with no sucess. I also had a meeting with Xusheng of our research group and he gave me alot of help on my game and that same MapEngine method. He had given me a way to try to make Pex understand the code better and search better paths by constraining the variables used in the factory methods.

Beginning my first paper

The past two weeks I've gathered my current information in order to find a topic for me to discuss for the SIGSCE student research project. I met with Justin Gorham for some help with Pex on how to use it with my game since he worked with Pex over the summer with the government. I also had help from Bellanov Apilli to learn how to use LaTeX. I had alot of problems just setting it up but got it working later on.

My topic of the paper was on how to deal with complex objects within a game using Pex. I have an example of this which I discussed in an earlier post (the MapEngine) that I will use as my example in the paper.

Continuation of Studies in OOP

My most recent research has been conducted with Charles regarding Object Oriented Programming. Primarily, I've been solidifying my knowledge in Object Oriented Programming, particularly in C# by consistently differentiating between the various aspects which compose a simple program.

Our most recent meetings had me perform several assignments of deciphering coding created by Charles, and explain the capabilities of each class. Inversely, I was given the instructions to create the coding for some assignments based on classes that Charles had defined.

The most prominent (and longest) example involved me creating the portion of a video game which assigned the values of location and velocity to two player characters.

I believe I had taken pictures of this example, but I am having difficulty locating it. When I find it I will upload it into this post, as it was particularly lengthy and will be trouble replicating. I will also ensure that my following posts will contain visual references to better visualize my experiences.


Take Care.

iTutor Improvements

The feedback compiled from the iTutor demo is as follows:

1. iTutor accepts three classes as inputs: Teacher solution, Student solution, and an interface common to them. The users of iTutor can be relieved by removing the interface as input. This interface can automatically be inferred from the public methods of Teacher solution. Also, before processing these solutions, iTutor needs to check whether it can find equivalent public methods in the student solution, if not, it can flag error.
2. iTutor currently renames the solutions to Student and Teacher. This can be easy if there is a single class. But situation can be difficult if there are many classes. Bellanov needs to investigate and develop features to address this issue.
3. Currently, the length of method-call sequences in tests has been limited to three. This can be made configurable.
4. Add "assertions" to all case statements, this would help in automatically generating test cases with variable number of sequences (as assertions exist right after the method-call invocations)
5. Make the index used in the switch statement as symbolic and remove it as a parameter for the driver method
6. Remove arguments passed to method calls as parameters from the driver method.
7. When the methods under test accept non-primitive types, a deep copy of the argument should be made. If not, the method under test of teacher solution may modify the argument before passing it to student solution. Similar techniques need to developed for comparing non-primitive return types.
8. A problem arises where methods can be overloaded, leading to multiple possibilities when instantiating each solution. iTutor shall generate numerous test drivers, each of which focuses on one constructor.
   
9. The logging feature that currently uses print line statements shall be implemented to send that information to a file, which can be accessed and analyzed later.
   

For the next week, I shall address as much of the following as possible. I predict that I will be able to cover and address points 1, 3, and 5. First and foremost, I am developing a test suite to properly test all of iTutor's present functionality to ensure correct behaviors. After I am convinced that the tool is in proper shape, I shall move on to address the points above.

Going forward

After having a meeting with the graduate students, and Dr. Xie. I need to write better PUTs since my skills on that area is lacking. I need to start out at unit testing and then from there do a PUT because without knowing how to write a correct PUT, I may never find anything of interest on how to test game applications using the XNA framework. I also need to get my research game Tears on graduate student computers so that I may get further feedback from them and continue to teach Khiry Object Oriented Programming. Above anything else, I need to find XNA specific research areas, so far I have been unproductive in finding anything of interest that is specific to the XNA framework, so that is my goal going forward.

Scurvy.Test and XNA

(August 2nd - August 15th)

Another part of the summer I worked with a Test Framework called Scurvy. It's an open source framework that lets you do unit tests specifically for the XNA framework. I was able to test that a loading function worked. The thing is that loading with XNA is different from different measures of loading because you have to use the XNA Content Pipeline. Well you don't necessarily have to use it but it would be in your best interest if you do to keep production of your game fast and easy and reliable. Here is a simple example:


Now it doesn't seem like much with this test but you will be able to get the correct result with this framework. I've tried other frameworks, including Pex, and .NET unit testing tools and they cannot get this simple test to work. I believe that with a little more digging around with Pex and Scurvy, I can find something that could be interesting with testing game applications using the XNA framework.

Pex and XNA cont'd

(July 19th - August 1st)

During the second half of my I started using Pex with Tears (my personal video game). Continuing from my last post on XNA and Pex, I discovered a problem with objects that are very complex (my MapEngine object) and testing my collision detection algorithm. In my game, which is a 2D platform game like Super Mario Bros. or Sonic the Hedgehog, collision detection and response is described as a way to keep all entities within a "world", to be able or not go through objects/areas that are labeled as such. So for the next couple of weeks I tried different possible ways in order to get Pex to work with this hardship and work with Scurvy.Test to test XNA specific objects.

*Update*
After meeting with Xusheng, I discovered that if you can make your PUTs smaller and constrain Pex a little, you may be able test these objects. Why do we have to constrain Pex in order to do this (and when is it necessary to do it), I don't know but that is another research topic.

Game Engine Demonstration and More C# Practice

After having made a request with my teacher, Dr. Michael Young, I was allowed to demonstrate the capabilities of one of my favorite non-program based engines, RPG Maker 2003. I was allowed to demonstrate the differences between it and Game Maker 7, by primarily comparing the organization of the sprites (charsets), variables, switches. I demonstrated the abilities which are intended for by the engine, (which is to create an RPG obviously), but also displayed a few of my games which were centered around Platforming and Third-Person Shooter styles.

Although it is often argued that non-code based Game Engines are detrimental to the growth of a game designer, I argue that these engines are in fact beneficial to a degree. I believe that they can lay the foundation for a smoother transition into programming, by introducing to the player the concepts which a game revolves around. It also allows an early programmer to experiment with their creativity and manipulate the various aspects of an engine (such as switches and variables in RPG Maker 2003) to create new methods of interaction. However, I will attest that a person who is seriously interested in becoming a Game Designer stray away from solely using these engines. The likelihood that they will create a mega-hit or controversial game is unlikely using an engine which anyone has the capabilities of using. Unless of course, you are Danny Ledonne.

I also began to differentiate between Console.Write() and Console.WriteLine with C#, and continued experimenting with the Console.ReadLine(). It can be used for entering text into the command line, and can also be used to separate lines.

Next Steps: JCute & Some Minor Adjustments

I met with Suresh and Kunal today, and we discussed the current implementation of the test driver, as well as the next steps. The concept of my test driver seemed to be mostly correct, and my next task involves incorporating JCute as per the sample code available on the website. After meeting with Dr. Xie, he exposed a problem in my implementation that pertained to inefficient code that would be costly in exhaustive situations. He proposed simply making the comparisons as the method sequences are generated and breaking out of the sequence once a failing test case has been encountered. For now, void methods have little impact on sequence but implementations using observer methods will be a future task.

Summarizing, my activities after the meetings are as follows:
1) Integrate JCute into the test driver, enabling it to choose all values, avoiding random testing
2) Converting test driver implementation to compare solutions during the method sequence generation, ending the sequence generation process when either a failing case or a completed sequence is encountered.

Outside of these tasks, I shall also begin the writing for this project.

The Beginning of my C# Tutelage

As a first semester Freshman, I believe it is my responsibility to not only maintain high levels of academic performance, but also make preparations for my designated career choice. These preparations will be done by learning programming languages from skilled programmers. Charles Woelfel has begun assisting me with C# programming.

Yesterday, October 16, 2009, we began with the reintroduction of Classes, Objects, and Methods. From my learning, I reviewed that Classes are basically a set of instructions which hold any set of instructions. Methods are the processes which are to be carried out in the Class. Objects seem to be objects, or items, which basically are a duplicate of another class or method, but renamed. These seem to be used most frequently in other classes.

My task for today, October 17, is to read a few more tutorials regarding command-line input. I'll also make preparations for another upcoming lessons by reviewing the information which I've learned, and attempting to create a few C# Programs of my own.

Method Sequence Generation

Throughout last week, I was working on the method sequence generation component of the test driver. I believe I am finished implementing this component, but I am still in the testing phases. I shall write and run concrete unit tests through this week. Some things I have been thinking about are intelligent ways on determining the method sequences. As of now, I am only thinking of generating the sequences randomly.

Outside the actual tool, I have received and began reading some papers in a similar field as my research. I am also developing the contents of the website pages I shall soon add to the course web. I hope to complete all, or at least a majority, of this during this week.

Working PDP

I have successfully built a PDP that is capable of using the RAdAC policies I have been working on. The only thing I don't like is that it is difficult to change the risk associated with a particular factor. For example, someone with Top Secret clearance is more trustworthy than someone with no clearance, but each level has a numeric value of risk associated with it. To change the numeric value, you must change the source. I'm thinking of storing the values of risk in a file that would be easy to edit or, even better, making a GUI. In any event, now that it is working I can begin working on the testing portion of the project. I will begin that this week.

Next Steps

When I've found time to work on the project, I spent it mapping out the series of steps required to implement this correctly. I met with Suresh to briefly discuss utilizing JCute. We decided taking an interface as input (or as one of them) may be beneficial, seeing as we cannot account for the student's possible helper methods. Using the interface to generate tests may be beneficial, since the behavioral differences span the methods defined in the interface. All that is required from respective solutions, outside the interface, are the constructors, which cannot be defined in the interface. Summarizing, this results in taking both solutions, as well as the interface being implemented. This results in 3 inputs. I shall post more information as I go on and implement this.

Back to Square One...

Today, I met with Yoonki in order to deploy and configure iTutor. The most recent version of the API we were able to recover, unfortunately, is the initial version. My next move is to discuss the limitations of the JPF tool with Suresh and Kunal. I recall problems in the previous implementation of the project, where I had the tool generating code that JPF would be unable to execute (i.e., switch statements). I will strive to quickly and efficiently have a working demo of the iTutor tool very soon, before my workload increases greatly. On a side note, I will also work on the tool's website, where instructions about using the tool will be available.

New Semester

I've updated the RAdAC project website. It now has everything that I've done so far. Last week I prepared my abstract for publication. Apparently I used the wrong template and I had a few grammatical errors to correct, but that is finished now and is submitted. I'm excited about starting a new semester. Between classes and research I will be busy! We had our first group meeting of the semester Friday. It was nice to meet everyone (some were new to me) in the group.

Fall 2009 Begins!!!

Today, the undergraduate and graduate students met with Dr. Xie for our regular group meeting. We were debriefed as to our summer activities and planned the projects that we will pursue in the future. It was decided that I and Kamar pursue the iTutor (name's probably going to change) project under Kunal's guidance. I briefly recall problems we ran into pertaining to JPF and hope that we shall resolve them this time around.

My initial task is to come up with somewhat of a prototype to display to anyone interested in the tool's functionality. Although it may not be a perfect exhibition of the final product, having something prospective to show is good enough for now.

Low code coverage doesn't always mean low test confidence

Code coverage is not always the best indicator of a test suite's usefulness, expressiveness, etc.

Contrary to the popular belief, there are cases in which high test coverage is not a good measure of test quality. In fact, high test coverage is not always possible.

Consider the following situation:

public class Container {
object a_1;
object b_1;
object c_1;
⁞
⁞
object z_1;

public object A_1{
get
{
return a_1;
}
set
{
a_1 = value;
}
}
⁞
⁞
⁞
⁞

}

public class A {
Container container;

public A (Container c)
{
container = c;
}

/*all methods of class A interact with container.A_1 ONLY*/

}

The other lettered classes follow the same format.
When Pex tests the Container class it will more than likely get 100% test coverage.
When Pex tests the lettered classes, however, it will not get 100% code coverage. The test coverage report includes coverage of the Container class, which does not require 100% coverage to test the lettered classes. The classes could arguably be written so that they are more easily testable; however, I did not write this code. I actually observed code similar to this in a real application. The lettered classes get 100% block coverage themselves but overall block coverage is never 100%.

Policy Rework and XACML Inefficiencies

I've had to rework the policy format I made earlier this week, mainly because it would have required an excessive amount of Java methods to make it work. I've decided to stay away from adding to the XACML specification wherever possible (with a few exceptions), because it cannot support all the possible rules that could occur in an RAdAC policy. Instead, only individual risk/need factors are calculated by Java methods; how they are combined is determined by the XACML policy. I did run into one problem with this approach, which was that the policy needs to use the risk and need measurements multiple times throughout the policy to make a decision. It is very inefficient to copy-and-paste the part of the policy that calculates risk and need each time they are used. I found a solution in the XACML 2.0 specification, the <VariableDefinition> and <VariableReference> tags. They store the result of a calculation for use throughout the policy. The only snag with using this, if we decide to go this route, is we will have to use the CVS version of Sun's XACML implementation to get the 2.0 features, which, according to their website, is stable.

Design Document

This week I've made a simple XACML template that contains the information needed by an RAdAC policy. I have yet to find a paper with concrete details on how anything dealing with RAdAC should be done; I've only found high-level descriptions of it. I think as long as it implements some mechanism to allow "operational need" to trump "security risk" it falls under RAdAC. Most of the papers I've read discuss multiple policies converging to determine risk, need, and ultimately a final decision. I am using a simple model that describes the basic functionality of RAdAC using a single policy. I have started working on a design document and will finish it soon.

Pex and XNA

(July 12th - July 18th)

Had another meeting with Dr. Xie about the unit testing using Pex with my game. I didn't have a good test (I need to write more unit tests with Pex to get tightly controlled test). Over the week I kept looking at the code and it seems that the way that the parameter is used and the way Pex uses parameters seems to have some conflict.

The class/method is:
public class Entity
{
...
public void handle_collision(MapEngine e)
{ ...}
...
}

The method basically handles the collision of the Map and the entity using it. Inside the method, if the position of the entity's collision box intersects with the a "wall" on the map, the entity's collision state changes to being hit and the entity's position is moved.

Now the problem is that the MapEngine object is only used as a reference for the character to see where it is on the map. It doesn't "change" per se, it only changes which map is the current map (and it has references to the previous and next map if any). The map itself doesn't change, the thing that does, is the entity's position. Depending on the position of the entity on the current map, it will collide or not, not depending on the current map.

The problem with Pex is that it uses different inputs to get high code coverage. In this case, the MapEngine is pretty much static in that the map is not changing everytime the method is called but the position of the entity is (and it is automatically changed in another method). If the input was the position value, then Pex would be able to test it. In order for collision detection in my game to work and for Pex to analyze it, I would have to change the implementation significantly. I am going to write up a thorough explanation of my problem to Dr. Xie next week and also test to see if there is anything I could do without having to test just in case I missed something.

Unit Testing with XNA continued.

(June 28th - July 11th)

I met with Dr. Xie and we reviewed on what I've been working on in the previous weeks. Instead of looking at the overall picture of Game Testing Automation, I'll focus more on certain things that game application use with xna and try to find anything that is an area of we interest like why can't we use certain things (Pex) for xna games in certain situations and write up why we can't. I did create a unit test that didn't use any content loading in my game that passed just as any other application. Earlier I had a problem with doing unit testing anything that used the Content Pipeline to load content for my game Tears (independently made but just used for researching purposes), I found a framework called Scurvy that alleviates that problem. I will retry previous tests that included content loading to test it out. Dr. Xie also recommended to look at GUI testing automation techniques to see if it's applicable to game applications.

Novaleaf Software & Unit Testing with XNA

(June 14th - June 27th. Late)

I interviewed Jason Swearingen, founder and president of Novaleaf Software (http://games.novaleaf.com/). His company tried model-based testing on his games but felt that it was wasting time and that he would rather just design the game and then go from there. Not really helping me at all. I tried to do some unit testing with XNA using Visual Studio Team System 2008's integrated Unit Testing software. I couldn't get anything that used loading content to work properly. Did some more debugging on my PongGame using NModel, still a little confused on the use of "features" but still working on it. Tried Pex a little bit but didn't spend expensive time with it. I will in the next two weeks. Going over and fixing some grammatical errors to my paper for Dr. Xie.

NModelling and Dishwasher

(May 31st - June 13th, late)

I continued to search for articles on anything related to game testing automation or any automation techniques that are used in the game industry to see why they used that technique or why they couldn't use another. Had to do some debugging on the GameModel program I made with the NModel Framework. I used that program to start implementation of a Pong game so that it will be easy to fill in. NModel has an aspect of it's framework called "Features" where you can add or take out parts of the program in a modular fashion. Just have to reread the chapters on it to get a good grasp on how to implement it since their code didn't seem to exactly explain on how to use it.

I also interviewed the creator of The Dishwasher: Dead Samurai, which was made with XNA. It's a Microsoft published game. James Silva, the creator, didn't use any automated test techniques, just the regular manual testing. I'm having a hard time finding any information on automated game testing techniques anywhere online but will continue searching. I started my paper for Dr. Xie on what I have been looking for.

First Post

(May 17- May 30)

So this is my first post concerning testing in video game application using Microsoft's XNA Framework. Dr. Xie recommended me to look at and read Model-Based Software Testing and Analysis with C#. It basically tests the model of your application and see if it can finds an errors like dead states and live states . They developed their own framework called NModel so that you can test out your application. I implemented a very simple application called LightSwitch, with a simple on and off state to get familiar with the framework. I then implemented a control flow of the game loop within a XNA developed application. I also contacted Shawn Hargreaves of the XNA Development Team at Microsoft for some information on any (if any) game testing automation techniques.

XACML and RAdAC

This week I have been familiarizing myself with XACML. It seems to be a very flexible policy language; albeit a bit complex. I've found the programmer's guide for Sun's XACML implementation to be a good introduction, but I'll be skimming the OASIS documentation to get a more thorough grasp of the standard. I am to use XACML to implement an RAdAC policy. To do so, two metrics will have to be calculated: a 'Security Risk' and an 'Operational Need'. I'm sure they will be calculated by some sort of function.

Paper Completion and OOPSLA

I've finished writing my paper. Last weekend was spent converting it to the ACM Proceedings format and reducing it to two pages. After reading Dr. Xie's PowerPoint, “Common Technical Writing Issues,” I found many grammatical mistakes in my own writing. It was very helpful in getting the review process started. After I addressed Dr. Xie's comments and fixed the errors discussed in his presentation, I submitted it to the OOPSLA Student Research Competition; I also submitted a poster describing the approach discussed in the paper.

Working Implementation

I've written a series of batch files and shell scripts to automate the testing process, so I now have a working implementation. The output is a series of text files (each corresponding to a partially masked rule) consisting of packets and their associated decisions. The number of packets generated depends on the number of rules involved in the masking, but on the policies I've been testing it with, they seem to be relatively low in number. I'm confident that the algorithm I am using to generate the packets is thorough. Most of this past week, I've been catching up on writing and doing classwork.

First Post

This is my first post in the REU blog. I received JeeHyun's tool from his SRDS08 paper this week. So far, I've been able to modify his tool to generate packets that target rule intersections. I've got to work a bit on making the output of the FireMAN tool useful to JeeHyun's tool though. This could be a bit awkward, since they run on different platforms (JeeHyun's tool uses a constraint solver by MS, simple Linux replacement available?), and use different firewall formats. In any event, paper writing is definitely more difficult than I originally anticipated. Now that I have everything I need (and know how it all fits together), hopefully the writing will become easier.

Semester Conclusion and Follow Up

At the conclusion of the spring semester, I came home greatly anticipating my grades. I ended up doing fairly well, receiving score of either an A or A- in all of my courses. Although my GPA went up quite a bit, it wasn't as much as I had hoped. Overall, I think I did well for myself.

I met with Dr. Xie to further discuss my work and the critical importance of citations. I learned it is very simple to plagiarize, even if it is unintentional. Regardless of intentional or unintentional plagiarism, it is still very wrong and carries dire consequences in research. Another issue that was addressed was communication. I will strive to improve and keep Dr. Xie and the graduate students informed of my activities.

We also discussed the research I shall be conducting this summer, deciding on my looking and familiarizing myself with the Pex tool. Thus I have been looking over C# and Pex tutorials since I have not done a lot of programming involving the two. I am constantly talking with Chris as far as the tool and his research is involved. The tool is very interesting and I hope I can apply it well in my summer research.

Outside of all this, I am updating my personal documents (i.e. my resume and website). Thinking back on all I have done at state, I seem to have done a lot outside of research as well. I shall compile all of my activities in an updated resume and ask for Dr. Xie's and the graduate students' opinions. The same applies for the website, although my summer research shall also be documented on it. I shall be making posts to this blog linking to my website about my progress or interesting findings I come across during my summer research.

STARS Banquet

At the STARS banquet, I presented a short overview of the research on Combinatorial Testing. I was glad when Dr. Xie said I did a good job and also commented on my flaws. The main flaw was I did not focus on the selling point of our approach, which is the fault-detection capability. The presentation would have been a lot better if I had made that point clearer. Overall, the banquet went very well and I met Ken Tate, one of the Director of Development & External Relations in the Department of Computer Science. This banquet served as a great networking opportunity, but unfortunately, I left my business cards in my room.

Readings & Poster

I have been reading further in the book on pragmatic thinking and am trying to apply its teachings towards aspects of research, (i.e. the paper and poster). I am still in the process of revising the paper, focusing on its lack of logical flow in order to improve it. Although I have the content in the poster paper, I am still plaguing myself with questions as far as converting the information into an actual poster. JeeHyun has been a great help and, with his help, I'm sure we shall create a great poster to present at the Tapia 2009 conference.

Functionality of Driver

I have a question regarding how the tool JPF would operate on the test driver I am writing.

Referring to the sample code Suresh sent me...

static testDriver(){
sequence generation;
}

One of my supposed testMethods...

testAdd(){
testDriver()

conduct testing operations & report differences
}

My question pertains to simply calling the testDriver method in each testMethod in order to generate the sequences, as opposed to reproducing the same code found in the testDriver in each and every test method.

iTutor Approach & Other Ideas

I've compiled my thoughts on the approach and began writing the paper. This section includes my understanding of the tool's functionality as well as possible routes the project may take, if feasible. I have also included a file which contains what I perceive as an example test driver and emailed it along with the paper to Suresh & Kunal for their approval.

I have also picked up the Pragmatic Thinking & Learning text Dr. Xie put on the shelf last week. I shall begin reading this weekend, as I wait for a response.