CMSC216 Project 4: A Shell Called Shellac

This Problem will have you complete a basic interactive loop for shellac_main.c. You'll also need to at least create the other files like shellac_job.c and shellac_control.c but need not put anything in them until the next problem.

The basic interactive loop for Shellac is similar to interactive command loops we have previously worked on such as:

• Project 2's Treeset Application
• Discussion 02's List Application

Review these past code bases as needed or get help from staff if you need assistance in setting up the basic Shellac main loop. Structure the interactive loop as follows.

1. An indefinite (likely while()) loop
2. At the top of each loop, print the (shellac) prompt
3. Read a line of text using the fgets() function (see notes on this later) and possibly print it if echoing is enabled
4. Tokenize the string using the provided tokenize_string() function
5. Analyze the 0th token for one of the built-in commands like help or exit and act accordingly
6. If the token is not a builtin command, start a child process to execute the job (completed in Problem 2)
7. The exit command breaks out of the interactive loop and shuts down the program.

Like the previous interactive programs that maintained a tree, hash table, or linked list, Shellac also maintains a data structure. It is a shellac_t struct which has an array of job_t structs to track processes launched in the background. Problem 2 will introduce those types while for now, just setting up the interactive loop is sufficient.

This section provides a sample of how the Shellac interactive loop should look and work once the problem is complete. It can be used as a reference and aspects of it appear in the test cases.

>> shellac
(shellac) help
SHELLAC COMMANDS
help               : show this message
exit               : exit the program
jobs               : list all background jobs that are currently running
pause <secs>       : pause for the given number of seconds, fractional values supported
wait <jobnum>      : wait for given background job to finish, error if no such job is present
tokens [arg1] ...  : print out all the tokens on this input line to see how they apper
command [arg1] ... : Non-built-in is run as a job

(shellac)

(shellac) tokens x y z
4 tokens in input line
tokens[0]: tokens
tokens[1]: x
tokens[2]: y
tokens[3]: z

(shellac) jobs
0 total jobs

(shellac) pause 1.2347
Pausing for 1.235 seconds

(shellac) wait 2
ERROR: No job '2' to wait for

(shellac) exit

The required help message for Shellac can be produced using the following, somewhat involved string.

void print_help(){
  char *helpstr = "\
SHELLAC COMMANDS\n\
help               : show this message\n\
exit               : exit the program\n\
jobs               : list all background jobs that are currently running\n\
pause <secs>       : pause for the given number of seconds, fractional values supported\n\
wait <jobnum>      : wait for given background job to finish, error if no such job is present\n\
tokens [arg1] ...  : print out all the tokens on this input line to see how they apper\n\
command [arg1] ... : Non-built-in is run as a job\n\
";
  printf("%s",helpstr);
}

Note the use of the Backslash character which allows continuation of a string in the next line of code. You may freely copy this function into your shellac_main.c.

As the help message indicates, you'll want to add cases for each built in commands. Samples of the behavior of each of these are in the sample runs but are outlined briefly here.

The C programming language is not known for its ease of handling string processing. Tasks that are straight-forward in other languages like splitting a string on spaces are tedious and error-prone in C. You'll get some sense of this with the need to get an entire input line and split it on spaces in Shellac.

Use the fgets() function to get whole input lines in the interactive loop. Its basic usage is as follows.

char *fgets(char dest[], int size, FILE *infile);

{
  FILE *infile = ...;
  char dest[64]; // a rather small size but...
  char *result = fgets(dest, 64, infile);
  ...;
}

• fgets() returns NULL if there is no more input which is an alternative way to cause Shellac to exit (no more input to read)
• If fgets() doesn't return NULL it will return a pointer directly to the input stored in the character buffer supplied, dest[] in the example above.

Shellac needs to analyze at least 0th space-separated string (token) on any input line separately. Additionally, some built-in commands need more than the 0th token while the need to start child processes requires completely splitting up the input line into space separated strings. This should be done using the provided tokenize_string() function from shellac_util.c. It's use is demonstrated in the function docstring:

EXAMPLE USAGE:
{
    char input[] = "gcc -o myprog source.c > output.txt";
    char *tokens[255];
    int ntok;
    tokenize_string(input, tokens, &ntok);
    // ntoks: 6;
    // tokens[0]: "gcc";
    // tokens[1]: "-o";
    // tokens[2]: "mpyprog";
    // tokens[3]: "source.c";
    // tokens[4]: ">";
    // tokens[5]: "output.txt";
    // input[] is now "gcc\0-o\0myprog\0source.c\0>\0output.txt"
}

Note that the tokens[] array points into the input[] array and the function modifies it. Later these strings will be needed separately from the input[] array so will copies will be made of them in Problem 2.

For now, the tokenize_string() function along with a printing loop in main() is sufficient to finish the tokens <arg1> ... builtin command.

One can exit Shellac using the exit command as in

(shellac) exit
>> 

Alternatively, in a scripted setting, one can simply stop providing input lines and Shellac should detect the end of input (end of file) and exit as well. In interactive settings, one can press the keystroke Control-d to indicate the end of input. The behavior should be to print a message and exit as in:

(shellac)      # press control-d
End of input

>>             # exited shellac and back to normal prompt

Use the return value of the input function used to get input lines in order to detect the end of input.

To be compatible with tests, Shellac must support command echoing as we have done in previous interactive programs (e.g. Project 2's Treeset and Discussion02's List application). When Shellac is invoked as shellac --echo with the echoing option passed, each interactive command is printed back to the screen. Some quick examples:

>> shellac     ## run without echoing
(shellac) help
SHELLAC COMMANDS
help               : show this message
exit               : exit the program
jobs               : list all background jobs that are currently running
pause <secs>       : pause for the given number of seconds, fractional values supported
wait <jobnum>      : wait for given background job to finish, error if no such job is present
tokens [arg1] ...  : print out all the tokens on this input line to see how they apper
command [arg1] ... : Non-built-in is run as a job

(shellac) exit

>> shellac --echo  ## run WITH echoing
(shellac) help
help               ## "help" command is echoed
SHELLAC COMMANDS
help               : show this message
exit               : exit the program
jobs               : list all background jobs that are currently running
pause <secs>       : pause for the given number of seconds, fractional values supported
wait <jobnum>      : wait for given background job to finish, error if no such job is present
tokens [arg1] ...  : print out all the tokens on this input line to see how they apper
command [arg1] ... : Non-built-in is run as a job

(shellac) exit     ## exit command is echoed
exit

>> 

Completing this problem will add the capability of running commands in Shellac. At its core, this is just a fork() / exec() combination. However, to lay the groundwork for more convenience in the shell, the functionality is broken into functions that manipulate two structs defined in shellac.h. Once these functions are complete, shellac_main.c can be modified to allow running a single job.

The following structs are defined in shellac.h and encapsulate the idea of a child command / job and tracking a collection of them in Shellac.

// job_t: struct to represent a running job/child process.
typedef struct {
  char   jobname[MAX_LINE];        // name of command like "ls" or "gcc"
  char  *argv[ARG_MAX+1];          // argv for running child, NULL terminated
  int    argc;                     // number of elements on command line
  pid_t  pid;                      // PID of child
  int    retval;                   // return value of child, -1 if not finished
  int    condition;                // one of the JOBCOND_xxx values whic indicates state of job
  char  *output_file;              // name of output file or NULL if stdout
  char  *input_file;               // name of input file or NULL if stdin
  char   is_background;            // 1 for background job (& on command line), 0 otherwise
} job_t;

// shellac_t: struct for tracking state of shellac program
typedef struct {                
  job_t *jobs[MAX_JOBS];         // array of pointers to job_t structs; may have NULLs internally
  int job_count;                 // count of non-null job_t entries
} shellac_t;

The job_t struct tracks a single "job" or command that is being run. The shellac_t struct contains an array of job_t structs so that several jobs can be run simultaneously in Shellac.

Functions that operate on the structs are in the shellac_job.c and shellac_control.c

Most but not all of the functions in shellac_job.c must be completed for Problem 2 so that jobs can be initialized, run, and updated as they complete. Below is an outline of the required functions. More may be added as needed.

// shellac_job.c: functions related the job_t struct abstracting a
// running command. Most functions maninpulate jot_t structs.

#include "shellac.h"

void job_print(job_t *job);
// Prints a representation of the job. Used primarily in testing but
// useful as well for debugging. Several provided utility functions
// from shellac_util.c are useful to simplify the formatting process:
// strnull() simplifies printing nice "NULL" strings and
// job_condition_str() simplifies creating a string based on condition
// codes.
//
// SPECIAL CASE: If the pid of the child is <= 0, print it just as 0
// or -2, or whatever negative number it is without the leading #
// (hash symbol).
//
// SAMPLE OUTPUT FORMAT: 
// job {
//   .jobname       = 'diff'
//   .pid           = #2378
//   .retval        = 1
//   .condition     = EXIT(1)
//   .output_file   = diff_output.txt
//   .input_file    = NULL
//   .is_background = 1
//   .argc          = 3
//   .argv[] = {
//     [ 0] = diff
//     [ 1] = file1.txt
//     [ 2] = file2.txt
//     [ 3] = NULL
//    }
// }

job_t *job_new(char *argv[]);
// Create a new job based on the argv[] provided. The parameter argv[]
// will be NULL terminated to allow detecing the end of the
// array. Allocates heap memory for a job_t struct and creates heap
// copies of argv[] via string duplication. The last element in
// job.argv[] will be NULL terminated as the paramter array is. The
// initial condition of the job is INIT with -1 for its pid and
// retval. The jobname is argv[0] and.  Normal foreground jobs have
// is_background set to 0.
//
// PROBLEM 3: If argv[] contains input or output redirection (> outfile
// OR < infile) or the background symbol &, then removes these from
// the argv[] and sets appropriate other fields. If problems are found
// with input/output redirection such as a ">" with no following file,
// prints an error and return NULL.
// 
// HINTS for PROBLEM 3: The provided array_shift() function from the
// shellac_util.c may prove useful to shift over input / output
// redirection though other methods are possible. Note that the ">"
// "<" and "&" strings are removed from the command line so must be
// handled wth care: either don't duplicate them or free() any
// duplicates of them before returning.

void job_free(job_t *job);
// Deallocates a job structure. Deallocates the strings in the argv[]
// array. Deallocates any input / output file associated with
// fields. Finally de-allocates the struct itself.

void job_start(job_t *job);
// Forks a process and executes the command described in the job as a
// process.  Changes the condition field to "RUN".
//
// PROBLEM 3: If input/output redirection is indicated by fields of
// the job, sets this up using dup2() calls. For output redirection,
// ensures any output files are created and if they already exist, are
// "clobbered" via appropriate options to open(). If input/output
// redirection fails, the child process should exit with
// JOBCOND_FAIL_OUTP or JOBCOND_FAIL_INPT.  If a job fails to exec(),
// the child should exit() with code JOBCOND_FAIL_EXEC. In this case,
// there is no need for the child process to attempt to de-allocate
// any memory as the OS will recoup all its resources on exit.

int job_update_status(job_t *job);
// Checks on the job for a status update. This utilizes a wait() or
// waitpid() system call. If the job has completed, updates its
// condition to reflect either EXIT or FAIL. For exits, uses macros to
// extract the exit status and assigns it the retval field.
// 
// PROBLEM 2: For foreground (default) jobs, blocks the parent process
// until the child is completed. Returns 1 for a condition change
// (e.g. RUN to EXIT / FAIL).
//
// PROBLEM 3: For background jobs, uses the WNOHANG option to avoid
// blocking the parent. If the job is finished, updates its retval,
// condition, and returns 1. If the job is not finished, just returns
// 0.
//
// For erroneous calls such as calls on a NULL job or on a non-running
// job without a pid, the behavior of this function is implementation
// dependent (may segfault, may exit with an error message, etc.) This
// situation is not tested.

Some of the functions in shellac_control.c must be completed as they would be used to add/update jobs in the main interactive loop. Below is a listing of the required functions that show up in unit tests.

// shellac_control.c: functions related the shellac_t struct that controls
// multiple jobs

#include "shellac.h"

void shellac_init(shellac_t *shellac);
// Initialize all fields of the shellac jobs[] array to NULL and set
// the job_count to 0

int shellac_add_job(shellac_t *shellac, job_t *job);
// Add a single job to the jobs[] array. Search for the first NULL
// entry in the array and add that on. Does basic error checking to
// see if array is full and prints an error message of some kind of
// so. Returns the index in the jobs array (job number) where the new
// job is added. If the jobs array is full, prints an error message
// and returns -1.

int shellac_remove_job(shellac_t *shellac, int jobnum);
// Remove the indicated job from the jobs array and replace its entry
// with NULL. Decrements the job count. De-allocates memory associated
// with the job via a call to job_free(). Does basic error checking so
// that if the specified jobnum is already NULL, prints an error to
// that effect.

void shellac_start_job(shellac_t *shellac, int jobnum);
// Starts the specified job number. First prints a message about
// starting the job of the format
// 
//   === JOB %d STARTING: %s ===\n
// 
// with jobnum and jobname filled in. Then uses a call to job_start()
// to start the job.

void shellac_print_jobs(shellac_t *shellac);
// Prints the job number and jobname of all non-NULL jobs in the jobs
// array.

void shellac_free_jobs(shellac_t *shellac);
// Traverses the jobs array and de-allocates any non-null jobs.

void shellac_update_one(shellac_t *shellac, int jobnum);
// Updates a single job via a cal to job_update_status().  If that
// functions return value indicates that the job completed, prints a
// message of the form
//
// === JOB %d COMPLETED %s [#%d]: %s ===\n
//
// with jobnum, name, pid number, and ending condition reported. Uses
// the job_condition_str() functon to create the condition string. For
// completed jobs, de-allocates them and removes them from the jobs
// array.
// 
// Examples of the printed message:
// === JOB 0 COMPLETED bash [#1000]: EXIT(0) ===
// === JOB 5 COMPLETED gcc [#22830]: EXIT(1) ===
// === JOB 1 COMPLETED cat [#22833]: FAIL(INPT) ===

void shellac_update_all(shellac_t *shellac);
// Iterates through all jobs in the jobs array and updates them. Used
// mainly to check for the completion of background jobs at the end of
// each interactive loop iteration.

void shellac_wait_one(shellac_t *shellac, int jobnum);
// Change the status of a background job to foreground
// (e.g. is_background becomes 0) then update that job to wait for
// it. Does basic error checking so that if the jobnum indicated
// doesn't exit, an error message of some sort is printed.

Completing this problem adds input and output redirection to the shell as well as the ability to run a job in the background, features most standard command line shells possess. It will make modifications to all the source files to affect this outcome.

Each of these is supported with the following syntax and they may be used in any combination.

1. Revisit job_new() to search argv[] for the symbols < and > followed by a file. The > or < and the file following it should be removed from the argv[] array for the command and have the output_file or input_file set appropriately. Here is an example from the tests:

      {
          char *args[] = {
            "ls", ">", "output.txt", NULL
          };
          job_t *job = job_new(args);
          job_print(job);
          job_free(job);
      }
      ---OUTPUT---
      job {
        .jobname       = 'ls'
        .pid           = -1
        .retval        = -1
        .condition     = INIT
        .output_file   = output.txt
        .input_file    = NULL
        .is_background = 0
        .argc          = 1
        .argv[] = {
          [ 0] = ls
          [ 1] = NULL
         }
      }
   

   Note how argc is only 1: both > and output.txt have been removed from argv[]. This removal can be done either during the copying loop from the parameter argv[] to the struct field argrv[] or afterwards. Careful to free any memory associated with extraneous strings like > which won't have pointers to them anymore.

2. Revisit job_start() and add cases after fork() has been called so that a child process re-arranges file descriptors. open() a file for reading on input redirection or writing on output redirection. Use dup2() to overwrite either Standard In/Out (or both) with the opened file.

3. When opening for writing, make sure to use options that will accomplish these there items

   • O_WRONLY: for writing
   • O_CREAT: which will create a file if it is not present
   • O_TRUNC: which will "truncate" any existing file to 0 size so that it is completely overwritten rather than writing bytes to only its beginning

   As well, use the version 3 arg version of open(fname, opts, perms) and set the permissions to S_IRUSR|S_IWUSR to enable further reading/writing of the created file.

4. Do some basic error checking: if a file can't be opened, detect it and exit the child process with either JOBCOND_FAIL_INPT or JOBCOND_FAIL_OUTP to indicate a problem with the redirection.
5. Check that the changes in the Shellac main loop have taken effect and then try the unit tests.

1. Revisit job_new() to check for the & background symbol. When detected, set the is_background field to 1.

2. Revisit job_update_status() to respect the is_background field

   • is_background==0 should block the parent on a waitpid() until the child finishes
   • is_background==1 should return immediately to the parent on waitpid() if the child is not yet finished. This can be accomplished with the WNOHANG option to waitpid().

   Return a 0 when a child job is not complete yet.

3. Complete any missing functionality in shellac_control.c for functions like shellac_add() so that multiple jobs can be added. As well, complete the shellac_update_all() function which waits for all jobs in the shellac struct. Use repeated calls to shellac_update_one() for this. Note that now job_update() may return a 0 to indicate that the job is not done yet in which case it should not be removed from the job array.
4. Add a call to shellac_update_all() at the end of the main() interactive loop so that at the end of each iteration, background jobs are checked for completion. This allows reporting on the completion every time another command is used or when just pressing "enter" to get another prompt.
5. Check that the changes in the Shellac main loop have taken effect and then try the unit tests.

Refer to the Project 1 instructions and adapt them for details of how to submit to Gradescope. In summary they are

Command Line Submission

Type make submit to create a zip file and upload it to Gradescope; enter your login information when prompted.

Manual Submission

Type make zip to create pX-complete.zip, transfer this file to a local device, then upload it to the appropriate assignment on Gradescope via the sites web interface.

You may wish to review the policy on late project submission which will cost 1 Engagement Point per day late. No projects will be accepted more than 48 hours after the deadline.

https://www.cs.umd.edu/~profk/216/syllabus.html#late-submission