~
to show Consolehelp
or hit TAB
for a list of commands. Press
Ctrl-C
to hide the console.
This tutorial expands on the
Quake Console tutorial by using the GitHub REST API instead of a faked filesystem. The purpose of the tutorial is to show how to wire up Josh.PathHandler
and custom commands to a remote API.
Type ~
to activate the shell we will be discussed belowl.
You can explore the current repository's file system via the standard ls
and
cd
commands as well as take advantage of path TAB
completion. Additional commands are:
user
- show the current user's infouser username
- change the user to explorerepo -l
- list the current user's repositoriesrepo repository_name
- change the repository to explore (supports
TAB
completion)
branch -l
- list the current repository's branchesbranch branch_name
- change the branch to explore (supports TAB
completion)
You will be limited to 60 requests/hour by the API (where each console command may use multiple requests). If you authenticate via GitHub, you will have a more flexible 5000 requests/hour to play with.
The approach of this tutorial is to walk through the pieces required to wire up
Josh.Shell
to a remote REST API via asynchronous calls to produce an interactive command line interface by explaining the flow. While some code will be shown inline, the primary code reference is the
annotated source code with links to specific, mentioned functions throughout the tutorial.
The console is designed to always be in the context of a repository, so that there is no state in which commands like
ls
are not available. It initializes with the
sdether/josh.js
repository and after this the user can switch users, repositories and branches, while always staying in the context of some repository. That means that at any point in time, we will have a current user object, a list of all that user's repositories, and a current directory on the current branch as state. Changing users picks a default repository and branch to keep that state populated. Branches and current directory information are loaded on demand.
The available state looks like this:
{
api: "http://josh.claassen.net/github/", // the proxy we use for the GitHub API
shell: $instance_of_Josh.Shell,
pathhandler: $instance_of_Josh.PathHandler,
user: $current_user_object,
repo: $current_repository,
repos: $list_of_user_repos,
branch: $current_branch,
branches: $lazy_initialized_list_of_branches_for_current_repo
}
GitHub provides a
REST API, giving access to most of its data and functionality. We're just going to worry about read capability around repositories. Each repo is basically a file system which plays nicely into
Josh.PathHandler
's area of applicability.
The user object comes from:
GET /users/:user
We never fetch an individual repository, instead opting to fetch all at user initialization via:
GET /users/:user/repos
We do the same for branches, fetching all, lazily, once we need to auto-complete or list them:
GET /repos/:owner/:repo/branches
Finally, we fetch the current directory via
GET /repos/:owner/:repo/contents/:path
This is done for the root during repo initialization, and for path completion, cd
, ls
, etc. on demand.
The API returns json, which is perfect for us as well, but it is rather drastically rate limited without authentication. I.e. without authentication you will be limited to 60 requests per hour, while with authentication the limit is 5000 per hour. For this reason, we proxy all github calls through a simple node.js application that can handle oauth to optionally authenticate the use of this console. This application is outside the scope of the tutorial, but the code can be found on the josh.js github-authentication-backend branch.
In order for us to show the console, we have to have initialized a user, a repository and retrieved it's root directory. This is done after
document.ready
$(document).ready(function() {
setUser("sdether", "josh.js",
function(msg) {
initializationError("default", msg);
},
initializeUI
);
});
We call
setUser(user_name, repo_name, err, callback)
for
sdether and
josh.js, before setting the authenticated user as the current user and initializing the UI of the shell.
function setUser(user_name, repo_name, err, callback) {
if(_self.user && _self.user.login === user_name) {
return callback(_self.user);
}
return get("users/" + user_name, null, function(user) {
if(!user) {
return err("no such user");
}
return initializeRepos(user, repo_name, err, function(repo) {
_self.user = user;
return callback(_self.user);
});
});
}
This function follows the pattern of providing both an
error and
success callback, since once the shell is initialized we need to make sure that any action we take on its behalf does result in its callback being called with some value, lest the shell stop functioning. Unlike previous tutorials, we're now doing network requests and those will fail sooner or later. For this reason we need to make sure we always have a quick
err callback to stop the current operation and call the callback provided by
Josh.Shell
on command execution. We also need to make sure that we do not mutate the application state until we are done with all operations that can fail, so that the worst case is us reporting to the shell that the operation failed while our current, known good state is preserved.
All API access goes through a helper function
get(resource, args, callback)
, which is responsible for constructing the
json call and inspecting the response. For simplicity, all error conditions just result in
callback being called with null instead of a
json payload.
function get(resource, args, callback) {
var url = _self.api + resource;
if(args) {
url += "?" + _.map(args,function(v, k) { return k + "=" + v; }).join("&");
}
var request = {
url: url,
dataType: 'json',
xhrFields: {
withCredentials: true
}
};
$.ajax(request).done(function(response, status, xhr) {
// Every response from the API includes rate limiting headers, as well as an
// indicator injected by the API proxy whether the request was done with
// authentication. Both are used to display request rate information and a
// link to authenticate, if required.
var ratelimit = {
remaining: parseInt(xhr.getResponseHeader("X-RateLimit-Remaining")),
limit: parseInt(xhr.getResponseHeader("X-RateLimit-Limit")),
authenticated: xhr.getResponseHeader('Authenticated') === 'true'
};
$('#ratelimit').html(_self.shell.templates.rateLimitTemplate(ratelimit));
if(ratelimit.remaining == 0) {
alert("Whoops, you've hit the github rate limit. You'll need to authenticate to continue");
_self.shell.deactivate();
return null;
}
// For simplicity, this tutorial trivially deals with request failures by
//just returning null from this function via the callback.
if(status !== 'success') {
return callback();
}
return callback(response);
})
}
Most of this function is actually devoted to CORS and rate limiting handling, which is unique to us calling the GitHub API via a proxy located on another server. When dealing with your own API, calls will likely just be $.getJSON()
Also for simplicity, any initialization failures, just bail out via
initializationError()
. Once we have a user, we can call
initializeRepos(user, repo_name, err, callback)
.
Commands are added via Josh.Shell.SetCommandHandler(cmd,handler)
where
handler is an object with two properties, exec and
completion.
Josh.Shell.SetCommandHandler($cmd, {
exec: function(cmd, args, callback) {
...
},
completion: function(cmd, arg, line, callback) {
...
});
}
});
Unlike the callback pattern we used for
setUser
, Josh functions do not have a separate error handler. Since Josh interacts with the UI, it has no concept of failure -- it has to execute the callback to continue executing. It is up to the caller to deal with errors and transform them into the appropriate UI response. But it still gives us the flexibility to undertake asynchronous actions, such as calling a remote API and complete the function execution upon asynchronous callback from the remote call.
The user
command does not have
TAB
completion, since doing efficient tab completion against the full set of GitHub users is beyond this tutorial. Instead it expects a valid username for
setUser(user_name, repo_name, err, callback)
and renders the user template with the new current user.
If called without a username, we simply render the user template with the current user.
The
repo
command can show information about the current repository, change the current repository or list all repositories belonging to the user. It also provides
TAB
completion of partial repository names against the repositories of the current user.
Given no argument, we simply render the repository template with the current repository.
If the argument is -l, we render the repository list template with the repositories we fetched on user initialization.
Finally, the argument is used to try and look up the repository from the known repositories list. If that succeeds, we call
setRepo(repo, err, callback)
, which fetches the root directory to initialize the current node of
Josh.PathHandler
before changing the current repository to the one specified. Upon switching we again render the repository template with the now current repository.
The completion handler for the command simply calls
Josh.Shell.bestMatch
with the partial argument and a list of all repository names.
bestMatch
takes care of creating the completion object with the appropriate argument completion and list of possible choices.
The branch command either displays the current branch name, changes the current branch or list all branches for the current repository. It also provides
TAB
completion of partial branch names against the lazy initialized list of all branches for the current repository.
Given no argument, the command simply prints the current branch name. The -l argument renders a list of all known branches for the current repository, while an actual branchname as argument will cause the console to change its current branch.
Showing the list of branches uses
ensureBranches(err, callback)
to lazily initialize the list of branches from the API.
The completion handler for the command calls Josh.Shell.bestMatch
-- just like repo
completion -- with the partial argument and the list of all branches.
PathHandler
provides unix filepath handling. This works by abstracting the filesystem into two operations, getNode(path, callback)
and getChildNodes(node, callback)
. The former returns a pathnode given a path string while the latter returns pathnodes for all children of a given node. With these two all tree operations including TAB
completion can be accomplished by PathHandler
.
A pathnode is an opaque object in which we can track any node state we want but has to have two properties:
{
name: 'localname',
path: '/full/path/to/localname'
}
getNode
is responsible for fetching the appropriate directory from the API either by relative or absolute path. A path is considered relative if it lacks a leading /
. PathHandler
tracks the current directory/file node in PathHandler.current
which is used to convert a relative path to an absolutish path. Since we also support the standard file system .
and ..
symbols and the github API does not, we then need to take the absolutish path and resolve these symbols before passing the resulting absolute path to getDir(repo_full_name, branch, path, callback)
.
It is the job of getDir
to fetch a directory node via GET /repos/:owner/:repo/contents/:path
. This API call returns either an array of file objects for a directory or a file object in case the path points directly at a file. We only care about directories for completion and cd
, ls
, etc. so we ignore file results and build a pathnode for the directory like this:
var node = {
name: _.last(_.filter(path.split("/"), function(x) { return x; })) || "",
path: path,
children: data
};
where name is set to the last segment in the path and children stores the actual API results (which are lazily converted to childNodes for completion by function makeNodes(children)
The use of the github console is fairly limited, but it illustrates that wiring commands and file system behavior to a remote API is fairly simple. While we chose to create a custom vocabulary, we could have just as easily proxied calls to mimic git itself. Either way, Josh.js
is an easy way to add a Command Line interface for your existing API or for an API custom tailored to your CLI, allowing you to create powerful admin tools without providing access to the servers themselves.