Implementing learning

Several important notes for you to make your learning successful:

• You can implement the learning offline, outside of the spacesettlers system so long as you can read in your model back into your agent and use it.
• In order to learn anything, you will need to collect a LOT of data.  Use the initialize function to open a file handle, save whatever data you need for learning during the agent’s lifetime, and then use the shutdown function to close the file handle.

heuristic agents

Since we move back to the regular non-CTF environment, the heuristic agents go back to the original heuristics as described in projects 1-3.  

Multiple agents

While not required, you are allowed to have multiple ships this project if you want!

Extra videos requested by class

Some extra help for the learning with notes and videos!

• Link to the notes for the first few videos
• Link to the notes for the last video

extra credit

The extra credit opportunities for being creative and finding bugs remain the same as in Project 1. Remember you have to document it in your writeup to get the extra credit!  

How to download and turn in your project

1. Update your code from the last project. You can update your code at the command line with “git pull”. If you did not get the code checked out for project 0, follow the instructions to check out the code in Project 0.
2. Note: the directories for config files changes BACK to the files we used for projects 1-3 for this project!  As do the targets you will want to run in build!  Change the SpaceSettlersConfig.xml file in spacesettlers/config/heuristicCompetitive or heuristicCooperative to point to your agent in src/4×4. The detailed instructions for this are in project 0. Make sure to copy over a spacesettlersinit.xml in the src/4×4 directory so your agent knows how to start. In spacesettlersinit.xml change the line <ladderName>Random Client</ ladderName> to the team name you chose in Canvas.
3. Write your learning code as described above
4. Build and test your code using the ant compilation system within eclipse or using ant on the command line if you are not using eclipse (we highly recommend eclipse or another IDE!). Make sure you use the spacesettlers.graphics system to draw your graph on the screen as well as the path your ship chose using your search method. You can write your own graphics as well but the provided classes should enable you to draw the graph quickly.
5. Submit your project on spacesettlers.cs.ou.edu using the submit script as described below.  You can submit as many times as you want and we will only grade the last submission.
   • Submit ONLY the writeup to the correct Project 5 on canvas: Project 5 for CS 4013 and Project 5 for CS 5013
   • Copy your code from your laptop to spacesettlers.cs.ou.edu using the account that was created for you for this class (your username is your 4×4 and the password that you chose in project 0). You can copy using scp or winscp or pscp.
   • ssh into spacesettlers.cs.ou.edu
   • Make sure your working directory contains all the files you want to turn in. All files should live in the package 4×4. Note: The spacesettlersinit.xml file is required to run your client!
   • Submit your file using one of the following commands (be sure your java files come last). You can submit to only ONE ladder. If you submit to both, small green monsters will track you down and deal with you appropriately.

/home/spacewar/bin/submit --config_file spacesettlersinit.xml \
--project project5_coop \
--java_files *.java

/home/spacewar/bin/submit --config_file spacesettlersinit.xml \
--project project5_compete \
--java_files *.java

• • After the project deadline, the above command will not accept submissions. If you want to turn in your project late, use:

/home/spacewar/bin/submit --config_file spacesettlersinit.xml \
--project project5_coop_late \
--java_files *.java

/home/spacewar/bin/submit --config_file spacesettlersinit.xml \
--project project5_compete_late \
--java_files *.java

Rubric – Part 1 Due Nov 5 11:59pm

• First due date: Nov 5 11:59pm 10 points for project proposal

  • 10 points for turning in a ONE paragraph project proposal on canvas here

  • 0 points for not turning in a proposal

Rubric – Part 2 REVISED due date: Dec 311:59pm

• Learning
  • 30 points for correctly implementing the learning method that you proposed and got feedback on (if you were told to choose a different method, you need to implement the method you were told to adjust to).  A correct learner uses learning in a way to improve performance and learning will be demonstrated in the writeup (though the curve is graded separately) using a learning curve.  Learning code should be well documented to receive full credit.  

  • 25 points if there is only one minor mistake.  

  • 20 points if there are several minor mistakes or if documentation is missing.

  • 15 points if you have one major mistake.  

  • 10 points if you accidentally implement a learning algorithm other than what you intended and it at least moves the ships around the environment in an intelligent manner.

• Graphics
  • 10 points for correctly drawing graphics (or using printouts) that enable you to debug your learning and that help us to grade it. 
  • 7 points for drawing something useful for debugging and grading but with bugs in it
  • 3 points for major graphical/printing bugs
• CS 5013 students only: You must implement a second learning method and document it in the writeup 
  • 20 points for correctly implementing a second learning method and documenting with a learning curve and paragraph describing it in the writeup
  • 10 points if you implement it but do not give a second learning curve
  • 5 points for bugs
• Good coding practices: We will randomly choose from one of the following good coding practices to grade for these 10 points. Note that this will be included on every project. Are your files well commented? Are your variable names descriptive (or are they all i, j, and k)? Do you make good use of classes and methods or is the entire project in one big flat file? This will be graded as follows:
  • 10 points for well commented code, descriptive variables names or making good use of classes and methods
  • 5 points if you have partially commented code, semi-descriptive variable names, or partial use of classes and methods
  • 0 points if you have no comments in your code, variables are obscurely named, or all your code is in a single flat method
• Writeup: 30 points total.  Your writeup is limited to 2 pages maximum. Any writeup over 2 pages will be automatically given a 0. Turn your writeup in to canvas and your code into spacesettlers.
  • 20 points for collecting data and demonstrating learning using a learning curve (in the writeup).  For full credit, make sure you explain why it is learning or not learning (if it isn’t learning, you will not lose your points if you can explain WHY it is not learning)
  • 10 points for describing your learning method in a paragraph or two and explaining why you chose to demonstrate learning in the curve that you present (e.g. I graphed decision trees by the number of leaf nodes to show overfitting or I graphed regression by RMSE over iterations to show it lowered error over time.)