Phizzy things ...

Phiz 3 and its documentation
Phiz Project at Sourceforge
Phiz Source Code at Sourceforge

Saturday, September 26, 2009

Building a simple JSON API with Phiz3

This is an example of using Phiz3 in PHP to make a simple restful JSON API. The API's goal is to return a representation of the calendar month as a JSON object showing weekday names. The JSON object will be an array structure with the month's day used as a key and each day's value containing the weekday name.

  1. (preparation): Collect some requirements (i.e. the paragraph above has a description of the problem)
  2. (preparation): Jot down some ideas about the parts needed to solve the problem describe
  3. (coding): Now let's translate this into some unit tests
  4. (coding): Write code to pass the unit tests
  5. (accessment): Does this meet the requirements of step 1, if not repeat

Preparation

Restate the problem you're trying to solve

Let's restate the requirements described in our owning paragraph. We want a Web base API that will transform a date request into a representation of the month with corresponding weekday names.

Identify the parts you need to solve the problem and what might be helpful

  • We want a RESTful API - PHP's $_SERVER['PATH_INFO'] will give us the restful arguments to the API request. We'll do this in our event loop.
  • We want to be able to input a date - We could use a YYYY/MM/DD structure to represent the date in a RESTful fashion e.g. api/2009/09/01
  • We want to JSON output - In September 2009 the JSON representation might look something like {'1':'Tuesday', '2':'Wednesday', '3':'Thursday', ... '28':'Monday', '29':'Tuesday', '30':'Wednesday'}

Coding

Write some unit tests

Phiz3 comes with its own simple unit test object called UnitTest which is in the source file UnitTest.php. Phiz3 also has it's own simple unit test library. Since this is about Phiz3's since this article is about Phiz3's Unit Tests. The there are three things you need to know to use Phiz's unit test object.

  1. How to create your Unit tests sub class create your own test object
  2. How to add your tests
  3. How to instantiate your test object and display results

Creating your own test object

You need to include UnitTest.php and create your own object class by extending UnitTests. You can get a current copy of UnitTest.php here. Here's an example of the code you need to write:

That's it. You've defined your own object called TestsWeekdayAPI as a child of UnitTest. Now let's add some test code. If we look at our requirements list we see that we need to parse the RESTful command line so let's start there. We're going to create a test called testRESTful. It will look at a string like /2009/09/01 and extract the year and month. We add this test as part of our TestsWeekdayAPI.

Let's look at what going in. First a bit of magic. If the method you create in your UnitTest child starts with the word 'test' it will be available to run later. Next we create an instance of our object (don't worry that we haven't written it yet, we'll do that later). We create some test data in this case We're simulating the contents of $_SERVER['PATH_INFO'] as $path_info. We setup some expected results run our method and compare.

Now that we've defined a test let's add the code create an instance of our object and run the test.

There you have it we have a basic unit test written. Now let's run it. Keep in mind it should fail sense we haven't written WeekdayAPI yet. Assuming you saved the code above as /www/TestsWeekdayAPI.php and UnitTests.php is in the same directory you could run the command php TestsWeekdayAPI.php to see the test results. You should get results looking something like this:

Fatal error: Class 'WeekdayAPI' not found in /www/TestsWeekdayAPI.php on line 5

Now we can code first part of our WeekdayAPI object. From our tests we've know we're going to pass in the restful path then return a date. Like creating unit tests we're going to sub class the Phiz3 object. Since we're haven't created the rest of our tests let's put in some place holder code.

We now can run our tests again (e.g. php TestsWeekdayAPI.php). We should get something like:

Fatal error: Class 'WeekdayAPI' not found in /www/TestsWeekdayAPI.php on line 5

Wait that's the error I got before. What I forgot to do is add the line to the top of TestsWeekdayAPI.php to include the new file WeekdayAPI.php that I've just created. Our TestsWeekdayAPI.php should now look like:

Now let's run that again (e.g. php TestsWeekdayAPI.php):

Running testRESTful
Assert Failed: The method RESTful should return a date. ERROR: RESTful() not implemented.
 testRESTful failed.
 0 assertions passed.
 1 assertions failed.

Summary
0 tests passed.
1 tests failed.

Now our unit test is working (i.e. no PHP errors but not yet passing). Let's go ahead and write the rest of our tests we need for WeekdayAPI. We're going to need to test building the JSON for a specific month. Finally we'll need a test for seeing if the whole assembled API is working. We can add place holder code like we did with RESTful method. When we get done we should have unit tests which run but fail for everything.

TestsWeekdayAPI.php should look something like this:

You're skeleton program should look something like this —

Running the command php TestsWeekdayAPI.php should yield something like —

Running testRESTful
Assert Failed: The method RESTful should return a date. ERROR: RESTful() not implemented.
 testRESTful failed.
 0 assertions passed.
 1 assertions failed.
Running testJsonForMonth
Assert Failed: The jsonForMonth should return a JSON representation for Sept 2009
 testJsonForMonth failed.
 0 assertions passed.
 1 assertions failed.
Running testEventloop
Assert Failed: The event loop should return our final results of the API request.
 testEventloop failed.
 0 assertions passed.
 1 assertions failed.

Summary
0 tests passed.
3 tests failed.

Now you're ready to code the API itself.

Coding the API

What we've done with the unit test approach is make the problem less fuzzy and made some decisions early about how things will behave. It's not that everything is fixed in stone yet. At this stage, or even when you start coding your actual program you might find that you're assumptions about the tests were wrong. If so stop coding the solution and write some more tests. The goal of the unit test is to provide you a solid foundation, guide you on implementing your plans and verify that you've met your expectations. If you find your plans need updating (i.e. some of your base assumptions were wrong) feel free to rewrite your tests but be vigilant against two temptations - writing tests to match code you've already written and changing tests simply to have code pass. Both of those are a waste of time and effort. You want the process to flow from writing tests, testing and seeing the tests fail, writing code that passes the tests. Our API is a simple problem so we're probably pretty close our solution already and I don't expect to have to change my tests though I might want to add some more.

Let's implement our RESTful method first and get that working. You're solution might look like —

If we've built this right when running php TestsWeekdayAPI.php should yield —

Running testRESTful
 testRESTful OK
 1 assertions passed.
Running testJsonForMonth
Assert Failed: The jsonForMonth should return a JSON representation for Sept 2009
 testJsonForMonth failed.
 0 assertions passed.
 1 assertions failed.
Running testEventloop
Assert Failed: The event loop should return our final results of the API request.
 testEventloop failed.
 0 assertions passed.
 1 assertions failed.

Summary
1 tests passed.
2 tests failed.

Now let's code jsonForMonth method —

We should have one more test passing then before —

Running testRESTful
 testRESTful OK
 1 assertions passed.
Running testJsonForMonth
 testJsonForMonth OK
 1 assertions passed.
Running testEventloop
Assert Failed: The event loop should return our final results of the API request.
 testEventloop failed.
 0 assertions passed.
 1 assertions failed.

Summary
2 tests passed.
1 tests failed.

And finally our eventloop might look like this —

My final test results run with php TestsWeeklyAPI.php looks like —

Running testRESTful
 testRESTful OK
 1 assertions passed.
Running testJsonForMonth
 testJsonForMonth OK
 1 assertions passed.
Running testEventloop
 testEventloop OK
 1 assertions passed.

Summary
3 tests passed.
0 tests failed.

Notice that I've decided to change what the eventloop returns. That is because we'll want to send output to the browser making the request. So I've decided that I'm returning a HTML div with any errors I've discovered rather then what boolean like I did with RESTful and jsonForMonth methods. It's not a problem because in testing for correct operations I wanted a valid JSON array and that is what I was testing for. Now we're ready to wire this up for an actual API that returns the month with days of the week as a JSON array.

Wiring up the final application

It turns out to be pretty simple. We've modeled how the API is supposed to behave all we need to do is set a content type and return results. Here's what I cam up with —

If you went to a URL where this was installed the output for /2009/09/06 would look like —

{"1":"Tuesday","2":"Wednesday","3":"Thursday", "4":"Friday","5":"Saturday","6":"Sunday", "7":"Monday","8":"Tuesday","9":"Wednesday", "10":"Thursday","11":"Friday","12":"Saturday", "13":"Sunday","14":"Monday","15":"Tuesday", "16":"Wednesday","17":"Thursday","18":"Friday", "19":"Saturday","20":"Sunday","21":"Monday", "22":"Tuesday","23":"Wednesday","24":"Thursday", "25":"Friday","26":"Saturday","27":"Sunday", "28":"Monday","29":"Tuesday","30":"Wednesday"}

Things to improve

First the error messages are pretty bad. Returning something more meaningful would be a big step in the right direction. Also error condition handling is very minimal and in a real API you'd beef that up. Our RESTful request could be simpler if we only required a year and month since the date isn't needed. You could extend the API so that requesting a specific date return that weekday name while specifying the month and year only returns the month with all the weekday labels.

I've tried to keep the unit tests minimal since this is an example. I've only check for correct conditions. You'll get better results if you also test for boundary conditions. This is particularly true as you build more complex API. This helps to prevent changing code in one place which breaks something in another place. Also to keep minimize the length of this article I've not commented my code. In practice I always try to comment at least the functions. I then can use a document generator (e.g. Doxygen) to generate source code level API docs.

Happy Coding

Posted by Atomic Typewriter at 6:00 AM 0 comments Links to this post
Labels: , , ,
Reactions: 

Saturday, September 19, 2009

A simple database app with Phiz3 and Jaxer (part 1 of 3)

Jaxer is all about Javascript server side, client side and combining the two. It has been a topic of interest to me since early Fall 2008. After a busy year I think it is time to dive in and start building things with Jaxer and the Javascript implementation of Phiz3.

Tools needed

  1. Stand alone Jaxer or Aptana Studio
  2. Phiz3.js and UnitTest.js
  3. Web browser and text editor

Let's start exploring by building a virtual notebook. To keep things simple I'm going to model my virtual notebook on the three binders I remember using in school. I'm going to complete this task in three parts. First I'm going to build a model of interactions as a program API, then I'll wrap those as a RESTful service and finally add a font end that talks to the service. I'm going to use a test driven development approach.

A word of caution

The code included is intended for instructional purposes and shouldn't be let out of your development sandbox (e.g. the stand alone Jaxer I downloaded for my Mac or Aptana Studio). I've taken liberties in my examples to minimize the onslaught of code ignoring many best practices in regards handling database connection information, user authentication and validating user inputs.

Breaking down the problem at hand.

The basic metaphor is a notebook with pages. I can write on the pages. I can add pages at the end of the notebook. I own the notebook and don't want anyone else to write in it. I do want to let everyone read it.

Let's elaborate a bit more. I need a sign in and sign out mechanism. In the real world I'd probably use OpenID, Shibboleth or other secure authentication method but this is a toy application so I can just store a secret that I can check against. It's probably a good idea to only show the options of adding, editing or delating pages when I'm authenticated so I'll remember that when it is time to create a user interface. Everyone will need a way to page through the notebook so I need a start page, next page, previous page and a view page. OK got that. Now what's on a page? I need to think about that a little bit more.

A page as a place in the notebook. Pages are ordered. That could be done with a page number. Wait, then there needs to be a way to insert pages. If the pages are numbered then renumbering is required on an insert and thats going to make things a little more complicated. Let's say I can only add pages to the end of the notebook. That makes it simpler. But what are we putting on the page? Text. I can just type away. Might be nice to date the pages too in case I'm trying to find something. Trying to keep things simple let's restate our requirements as a list of activities &mdash

  • I need a way to authenticate so I can manage my notebook
  • I can add a page to the end of the notebook
  • I can remove a page
  • I can write on a page
  • Everyone can page through the notebook and read pages

Let's translate this into SQL pseudo speak for a moment —

  • A collections of pages, that's a table
  • A single page is a row
  • A row needs a unique id which is automatically increment for ordering
  • Each row has a page id, text content and a modified date

Remembering our authentication issue we also have another table —

  • A table of with one user — me.
  • In the single row I can store a pass phrase to restrict who can modify things. In real life I can replace this with a proper authentication mechanism.

Sounds like we have a good idea of how to represent the data in the problem. Let's sketch this out a little further as an API oriented list of things —

  • me
    • sign in
    • sign out
  • pages
    • (me) add page
    • (me) remove page
    • (me) update page
    • (everyone) view page
    • (everyone) next page
    • (everyone) previous page

If I think about having a RESTful API (or at least semi-RESTful) the URLs might look like —

  • notebook/signon
  • notebook/signoff
  • notebook/ — Show the most recent page added with navigation to earlier pages
  • notebook/PAGE_ID — Show a specific page in the notebook
  • notebook/PAGE_ID?page=next — get the next page
  • notebook/PAGE_ID?page=prev — get the previous page
  • Add, update, remove a page can be done with a POST to the service after authenticating:
    • notebook/?page=add &mdash create a new page
    • notebook/PAGE_ID with a POST &mdash replace a page
    • notebook/PAGE_ID?page=remove &mdash remove a page

I think we have a pretty good handle on the problem now. Next step, which might seem counter intuitive, is to create some tests to see if we've achieved our goals. I can do that by using Phiz3's UnitTest object.

Writing some test code

Setting up

To try out these example you'll need a text editor and Jaxer. I use SubEthaEdit with the stand alone Jaxer available from www.jaxer.org but you could also use Aptana Studio from the folks who created Jaxer. You'll also need the latest copy of UnitTest.js from the Phiz project at sourceforge.net and a copy of Phiz3.js. Once you have those we're ready to get started.

Create a folder for your project under Jaxer's public folder. On my Mac that's in /Applications/Aptana_Jaxer/public. I called my folder Notebook (i.e. /Applications/Aptana_Jaxer/public/Notebook). Create a new text file called TestsNotebook.js and a new html called TestsNotebook.html. Here's some boiler plate to get started.

TestsNotebook.js:

TestsNotebook.html:

Take notice of the extra attribute in the script tag i.e. runat="server". That's really important because this tells Jaxer to run the Javascript server side and this allows us to work server side with SQLite3 or MySQL or the filesystem.

The files in my working directory look like this —

  • UnitTest.js
  • Phiz3.js
  • TestsNotebook.js
  • TestsNotebook.html

Now let's put some placeholder tests together in TestsNotebook.js.

Notice that I have a test for each of the API actions I've described in my outline plus one to test the basic construction and setup of my database schema. I've used assertFail as a placeholder so when I run this tests I'll see them working but failing. Fire up Jaxer and point your browser at TestsNotebook.html. Your should see a result that looks like this —

If you've gotten this far we're ready to start building some real tests as we develop this application together.

Fleshing out an SQL Schema

First step is to make sure I can create, connect to and update my database. To keep things simple I'm going to use Jaxer's built in SQLite 3 database object. You could also use MySQL by changing the Phiz3 connection attributes (i.e. change db_type to mysql and add db_host, db_user, db_password attributes with Phiz3's set method). That's a feature of using Phiz3 instead of Jaxer's native DB object. Here's the steps I need to take to flesh out testSQLSchema and get going &mdash

  • Create a Phiz3 object and configure it for the database connection
  • Open the database connection
  • Write an SQL statement to create our database if they does not exist
  • Execute an SQL DELETE, INSERT, UPDATE or REPLACE statement to make sure I can modify the tables.
  • Add me to my authentication table.

Start by modifying testSQLSchema to look like this —

Here's the MySQL version of the code for testSQLSchema if you want to go that route —

You'll want to replace the db_host, db_name, db_user, db_password with something appropriate for your environment. From here on out with the exception of a few little SQL dialect differences between SQLite3 and MySQL the Javascript should all be the same for either database platform. If you run your tests now you should get something like this —

Building and testing our SQL Schema

We've added the database connection now let's add some tables. Phiz3 provides a method called execute. It will execute an SQL statement and if successful return true otherwise return false. There is a caveat here. If you make a mistake in your SQL statement (e.g. forget to have the VALUES keyword in a REPLACE statement) Jaxer will catch it before Phiz3 has a chance to evaluate it. If so your app will stop processing and you'll need to look at your Jaxer and Apache logs to diagnose what is wrong (i.e. on my Mac they're at /Appplications/Aptana_Jaxer/logs/). Since my goal isn't to teach SQL here I'll just give you some code and gloss over the details —

If you reload TestsNotebook.html you should see that we're passing all the tests in testSQLSchema. If so then you're ready to go to the next step otherwise you'll need to do some debugging.

Unexpected lesson

I stumbled across this while writing. When I originally typed up my testSQLSchema method I used the MySQL style primary key definition in the create table statements. SQLite3 has a different syntax. If you use MySQL's syntax with AUTO_INCREMENT before PRIMARY KEY SQLite3 will happily accept the SQL statement but your primary key will not store the auto incremented value. But, and this is really important, the value it should have stored will be incremented. The difference between SQLite3 and MySQL is two fold. In SQLite3 you need to use AUTOINCREMENT instead of AUTO_INCREMENT and the ordering of the deffinition counts. In MySQL you can do something like my_id INTEGER AUTO_INCREMENT PRIMARY KEY NOT NULL in SQLite3 this should only be written as my_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL. If you write the MySQL version like my_id INTEGER PRIMARY KEY AUTO_INCREMENT NOT NULL and run it with SQLite3 then Jaxer will throw an error and you'll notice the difference in the keyword (i.e. AUTO_INCREMENT versus AUTOINCREMENT).

Implementing my Notebook object

Now that I have an idea about how I'll store my data in the database it's time to start implementing tests for my notebook object. testSignOn is a good next step. What would signOn do? Well it should check to see if the secret provided matched the one in the me table and then I can set a cookie variable to let me known when to allow adding, updating or deleting a page. Sounds good. I'll also need to create a new Javascript file where I'm going to define my object. I'll call that file Notebook.js. I will need to modify my TestsNotebook.html file to include that so it is available to my unit tests.

Updated TestsNotebook.html

Notebook.js

Notice that we haven't really implemented anything yet. We've just put a placeholder or stub for the function we'll flesh out in a minute. Here's the updated testsignOn —

After updating TestsNotebook.js we should get test results looking like this —

Now we're ready to do some real coding. By figuring out what we're going to test for we've already decided how signOn will work. We pass the big secret thingy and we return true if successfull or false if not. That means we need to query the me database table. Here's what I coded up to meet the test requirements.

Running the tests should yield results similar to these —

Wait I didn't test for the cookie status. Better add that now and see what happens. Remember I write the test first then I fix Notebook.js to conform to the test. Here's the updates —

TestsNotebook.js

Notebook.js

Now that we have signOn complete let's do the same by writing the test code for signOff method.

TestsNotebook.js

Notebook.js

Supporting our notebook pages

For addPage, updatePage, removePage, I need to keep track of whether or not I've signed in. Jaxer supplies a Cookie object for that (e.g. Jaxer.Util.Cookie.get('me') == true if sign in is successful). If I'm signed in then I can add, update a remove a page and that's just a matter of forming a SQL statement to send to the database. Phiz3 provides two ways to execute SQL. The first is execute(). The second is mapExecute() which allows you to write a simple SQL template and map in object attributes as needed. Phiz3 also provided a getRow() method which returns the row as an object. It's pretty straight forward.

TestsNotebook.js

Now let's look at the code that actually does the lifting.

Notebook.js

Wrapping up

We're left with the basic navigation functions for the notebook — start, next, previous, and view. Like add, update, and delete these are simple methods wrapping a SQL statements. Unlike add, update and delete we don't need to check if we're authenticated. Coding these up we follow the same process (i.e. write the tests first). Remembering what our navigation methods are supposed to do will guide us in writing those tests.

  1. testStartPage - this will select the most recently added page to the notebook, i.e. page with the highest primary key value.
  2. testNextPage - get the current key value and return the next one available. If you're on the last page return false.
  3. testPrevPage - get the current key value and return the previous one. If you're there are no more previous pages return false.
  4. testViewPage - make sure we getting the page we think we're supposed to get and that the content matches what we expect.

Let's code the tests up, code up the library and run our tests.

Complete TestsNotebook.js

Complete Notebook.js

Complete test results

Coming attractions

That wraps the first part of this article. Next we'll hookup the programming API as a RESTful Jaxer service.

Happy Coding!

Posted by Atomic Typewriter at 6:00 AM 0 comments Links to this post
Labels: , , , , , , , ,
Reactions: 

Saturday, September 12, 2009

Configuring Phiz3 Objects

(for PHP, Javascript Server Side and Javascript Browser Side)

Phiz3 supports both PHP and Javascript object configuration. In each language implementation there are four ways to configure a Phiz3 object. If you are working with PHP server side and Javascript clients side Phiz3 provides two additional PHP methods, toJSON() and toJavascript(), to assist in configuring your Phiz3 object client side. Here's the basics for Phiz3 objects server side —

  • the set() method. (e.g. in Javascript obj = new Phiz3(); obj.set("app_path", "/www");)
  • the constructor method. (e.g. new_object = new Phiz3(obj);)
  • Simple configuration markup.
  • JSON configuration markup.

The basic configuration file formats

Simple configuration format

Phiz3 provides a simple configuration file format to configure objects or applications. Configuration files are very helpful in making web applications portable between hosts. Phiz3's configuration format supports key value pairs and comments. The key and value is delimited by a colon and the value is terminated by a new line. A hash mark is used to start a comment which is terminated with a new line. Here's a simple example.

JSON configuration format

Phiz3 also supports using a JSON base configurations. It's relatively to type the above configuration in JSON format:

Type these two examples up and save them as myapp.conf for the simple configuration and the JSON version save as myapp.json. Either could be used to initialize a Phiz3 PHP or Javascript object.

Configuring Database connections

A cool thing about using Phiz3 objects with a configuration file is the ability to switch between SQLite and MySQL as the database target server side. Here's an example of going from SQLite to MySQL using the simple configuration markup.

SQLite version —

MySQL version —

Whether you're working server side with Javascript via Jaxer or with PHP switching between targets is easy as updating your configuration file.

Putting the pieces together

Integrating your configuration with your application code requires two things —

  1. Read your configuration file from disc as a string.
  2. Pass the string to the constructor of the Phiz3 object.

We're going to do this below and print out the results (NOTE: I've assumed your configuration files are in /www) —

PHP version

Javascript version (NOTE: I'm assuming this is running server side via Jaxer and myapp.conf is in /www)

A similar approach using a JSON file called myapp.json would be accomplished with —
Javascript with Jaxer works in a similar fashion as before (NOTE: I've assumed the configuration file is at /www/myapp.json).

So why is the useful?

Having a configuration file separate from your code base makes it easy to move your code from host to host (e.g. from your development box to your quality assurance box and finally to production) but more importantly it makes it really easy to move configuration information from server to client between PHP and Javascript. Here's some examples of integrating PHP server side with Javacript client side.

Using toJavascript() approach

In this example I'll read a simple configuration in PHP then construct a Phiz3 Javascript object from PHP which will be used to configure another Phiz3 object in the client page.

Using toJSON() approach

If you don't mind mixing PHP inside Javascript this approach works nicely.

Happy Coding
Posted by Atomic Typewriter at 6:00 PM 0 comments Links to this post
Labels: , , , , , ,
Reactions: 
Subscribe to: Posts (Atom)