Tsinghua University

Introduction to Distributed Systems
Spring 2013

 
 
Overview  
Syllabus  
Calendar  
Labs  
Homework  
Final Project  

Overview

The final project is entirely up to you except that you should do something related to the course material. Thus, you can do anything related to distributed systems. You are encouraged to build some things based on your fault tolerant yfs file system. It is OK to do something else but you should take care of the time limitation.

Requirements

  • You can do your final project in a team of no more than 3 persons. So, find your colleagues as early as possible. Please send me (cc to TA) the names (as well as the student IDs) of your team members no later than 12:00pm (TBD).
  • You and your team members should do something related to distributed systems. The programming difficulty should be similar to the yfs lab. Any extension to yfs should be OK. So, your team should check with us about the final project before you start.
  • You should hand in your proposal and report as mentioned below.
  • No delay (Hard deadline: TBD)

Proposal and Report

Besides doing your final project programming, you have to submit several documents for various purposes.
  • Final project proposal and plan(Deadline: TBD): Before you and your team members start doing your final project, your team should hand in the proposal and plan for you final project and discuss with us for approval. Some suggestions are given below.(You are not required to follow the guideline strictly.)
  • Final project report (Deadline: TBD) : After you have finished your project and before doing the final demo, your team should hand in the final project report. Some suggestions are given below.(You are not required to follow the guideline strictly.)

Final project proposal

You will submit a proposal for your final project including the following content. Make an appointment with us to discuss with your final project.
  1. Introduction
    short introduction to your final project such as the problem you want to solve, the ideas you want to follow, the usefulness or contribution of your project, the background of your project. You will have to list some interesting aspects of your final project. If it is not a new idea, you should include something that can show your improvement.
  2. Proposed solution
    Describe your solution to your problem. This will include system architecture, essential features and the organization. Some figures are helpful when you show the components of the system as well as the interaction among them. Show the difficulties and workload of your project and you solution to tackle them.
  3. Development plan
    The coarse level timeline for the project should be given. Some major milestones should be included with short descriptions for them. They should be used when you give your mid-report. Show the feasibility on accomplishing your project before the deadline.
  4. Team members
    The team members should be listed in the proposal. Each member should be assigned some work in the project and should be described in the proposal.

Final project report

Some contents of the final project report can be reused from the proposal. The report should tell the whole story. The report should include sections of introduction, challenges and design (with architecture), implementation, difficulties you have met and how to deal with them, experimental results and how to improve the system, as well as some experiences you've gotten from the project. Finally, the contribution of each member for the project should given in the report.

Final project demo

Final project review will be hold on TBD. Each team will be given 20 minutes to demo the final project. You are suggested to use some slides to do the presentation. The presentation includes all the things you think that other people might be interested in i.e. show your great work during the final project. The time is strict so you should prepare the cases that during the demo your system might crash.

Notes:

  • Things to avoid: we have learned how to build a distributed system. So, it is not allowed to define your final project on how to use a distributed system. For example, writing MPI programs or Hadoop programs are not allowed. However, using MPI or Hadoop to build a new system should be fine. You need to check the final project proposal with us.
  • If you want do something related to yfs, here are some good suggestions(from mit 6.824) you can follow.

Some Projects Ideas: (You can choose by your own and not required to follow the ideas below)

  • Online Chatting: You may want to build a chatting program. You might want to provide the group chatting capability. How can you reserve the causality between different messages from different users?
  • Distributed Key Value Store: You have learned how Dynamo from Amazon works. You may want to build your own. However, can you build a distributed key value store supporting the range search? Can you still use DHT as the overlay network infrastructure? You can choose any implementation of local store engine such as BerkeleyDB or Redis as your building blocks and only implement the distributed part of work.
  • Scalable Distributed File System Metadata Service: Google file system has a single master to store all the metadata. Can you distributed the file metadata to multiple machines? If the block size is shrinked to 1MB which means their might be huge number of middle size files such as small vedios and photos, there will be a lot of file metadata. How can you design a scalable metadata system contains over 10million files?
  • Distributed File Sharing: You put a file on a server and shared to users. However, some 'hot' files might be accessed simutaneously by a lot of users. How can you improve the performance?
  • Library Management: A library can have a lot of books. Books can be bought from the outside and made available for users. Users can borrow sevaral books from the library and return the books back to the library for other users. This, books can be considered as the shared objects for users. You might want to build a library management system to handle all these cases.
  • More interesting projects can be found from CMU Distributed System course.
 
2013 High Performance Computing Institute, DCST, Tsinghua