Tsinghua University

Introduction to Distributed Systems
Spring 2013

 
 
Overview  
Syllabus  
Schedule  
Labs  
Homework  
Final Project  

Labs

Course Labs take 40% of your final score. Our labs are borrowed from the distributed course from MIT 6.824. You can refer the course website go get the Lab Information.
In the sequence of labs, students will build a distributed file system called Yet Another File System (yfs). yfs uses one extent server to store the data and one lock server to coordinate the file server operations. The file server architecture will look like following:

yfs architecture

So, you will construct three processes as shown in the figure. yfs provides the file service using FUSE toolkit. FUSE will forward application file system operations to the process of yfs. yfs processes store the data in the extent server and all the client will see the same shared file system. lock server will be used to coordinate the operations of yfs processes and keep the file system consistent.

This is a series of labs for helping you to build a distributed file system. Each lab will give you a specific focus on the part of the file system. There are about 8 labs during the course of building the whole system. I will give you two weeks for each of the lab of the first 7 labs. And you are required to complete the final lab before the end of the semester. The lab guide can be gotten from the MIT web site. I have used the materials on their web site to construct 8 lab guides for your reference. Some of the information is specific for your course such as the lab server, so you should get such information in the pdf documents.

Lab Policy

The first 7 labs must be completed individually. You are not allowed to cooperate on those labs. You can discuss with the classmates on general topics of distributed systems including distributed file systems. However, you are not allowed to discuss with them about the details for completing the labs. You are not allowed to copy from others or from any other resources such as from the Internet. I will check the labs and ask you questions about the each lab. So, if you are not capable to give me the satisfying answer, the corresponding lab will be considered as fail. Anyway, if you can complete the work all by yourself, you should not worry about this issue. The last lab is defined by yourself and you can work as groups with 3 members at most. I will give the score for the 8th lab based on the relevance, difficulty, completeness of your design and implementation.

Detail Guideline

As we borrow the labs from MIT 6.824, you can refer to the guidelines in the lab section their course. However, there is some difference for the lab environment setup. I have copied the guideline here for each lab and did neccessary modification.
  • Lab environment setup
    1. You might have to read the design of the frangipani file system.
    2. Here is the website of FUSE. FUSE will be used as the programming environment and you should have your linux machine installed with FUSE module, library and headers.
    3. Detail guideline for setting your lab environment.
  • Lab 1 : Lock Server, Deadline: TBD.
    In this lab, you'll implement a simple lock service. You will have to implement the at-most-once semantic and your code should work under lossy network.This is the detailed guideline for Lab1. Notice that you do not send your result to mit 6.824 submit email.
  • Lab 2 : Basic File Server, Deadline: TBD.
    In this lab, you will start your file system implementation by getting the following FUSE operations to work: CREATE/MKNOD, LOOKUP, READDIR SETATTR, WRITE and READ. This is the detailed guideline for Lab 2.
  • Lab 3 : MKDIR, UNLINK, and Locking, Deadline : TBD
    In this lab, you will implement the handlers for MKDIR and UNLINK. You will also add locking to handle concurrent operations. This is the detailed guideline for Lab 3.
  • Lab 4 : Caching Lock Server, Deadline : TBD
    In this lab you will build a lock server and client that cache locks at the client, reducing the load on the server and improving client performance. This is the detailed guideline for Lab 4.
  • Lab 5 : Caching Extent Server + Consistency
    This lab will improve your file system performance by caching the file system data. However, the extents cached at different yfs_client should be kept consistent. This is the detailed guideline for Lab 5.
  • Lab 6 : Paxos
    You will have to implement a replicated state machine for your lock service. In this lab, you will implement Paxos and use it to agree to a sequence of membership changes. This is the detailed guideline for Lab 6.
  • Lab 7 : Replicated lock server
    This lab will replicate the lock services to multiple machines (RSM). Paxos will be used to decide the current view (members) of the lock servers. This is the detailed guildeline for Lab 7.
  • Lab 8 : see Final Project
    .
 

Notes

Lab1 to Lab7 Labs should be completed individually.

Lab reports should be submitted before the lab due time.

 
2013 High Performance Computing Institute, DCST, Tsinghua