A simple wiki using MVC and MySQL [WHIFF documentation]

Home
Info
- Demos
- Tutorials
- Quick start
- Syntax quickref
- MVC/wiki
- AJAX/calculator
- authentication
- Chat
- Charts
Concepts
Reference

WHIFF DOCUMENTION

A simple wiki using MVC and MySQL

This example implements a "wiki" interface. The implementation organizes the application using the "model/view/controller" (MVC) pattern within WHIFF and also illustrates how to add an SQL database as a resource to a WHIFF application among other things.

Contents:

The model/view/controller pattern
Configuring the application directory
The views standing alone
Adding stubbed models
Injecting views into models using controllers
Launching a test server with stubbed model functionality
More controllers
Configuring a simple file based storage resource
Unstubbing the model components to use the storage resource
Running the server unstubbed
Deploying the wiki as a CGI script
Configuring a mySql database storage resource
Configuring a root application with the mySql database storage resource
Summary

In this tutorial we implement a simple application ‡

×
Welcome to the obligatory wiki implementation.

in a pedantic way in order to illustrate a good way to organize more complex applications: the model/view/controller pattern (MVC). ‡

×
For this application the full blown MVC used here is overkill -- the architecture could be streamlined and made simpler and easier to understand. However, if this "proof of concept" were to grow into a "real" application the MVC architecture would make it easier to enhance and maintain the application as it grows larger.

In particular we implement a very simple wiki application wwiki which allows users to find discussions of concepts by the name of the concept, and also to edit the discussions or add new ones. ‡

×
See the Wikipedia wiki wiki entry for more information about wikis.

The source code for wwiki is available in the distribution at INSTALL/demo/wwiki. The application is also deployed as a CGI script at http://aaron.oirt.rutgers.edu/cgi-bin/wwiki.cgi/listPages -- please have a look!

The model/view/controller pattern

MVC is one way to organize the components of a user facing application. In the MVC approach the presentation of information (the views) are separated from the application data operations (the models), and controllers connect the presentation views to the model logic. The components of the wwiki application can be categorized as either views, models, or controllers.

Component Name	Responsibility
*Views*	Format HTML page presentations.
`formatPageList`	Show a list of summaries for all wiki pages.
`pageSummary`	Show a summary line for a single wiki page
`formatPage`	Display a single wiki page.
`editForm`	Display an HTML form allowing the user to add or edit a page.
*Models*	Perform operations on application data.
`getItemList`	Get summary information for all wiki pages.
`getItem`	Get wiki page data using the name of the item.
`addItem`	Change or add wiki page data for a named paged. Validate the data and fail if the data is not complete.
*Controllers*	Connect user interaction with views to model actions.
`listPages`	Get page summary data using `getItemList` and format the summaries using `formatPageList`.
`viewPage`	Use `getItem` to find page information and present the information using `formatPage`.
`addPage`	Format a blank form for adding a page using `editForm`.
`editPage`	Use `getItem` to find page information and present a form to edit the data using `editForm`.
`changePage`	Take submitted form data from `editForm` and attempt to add or change a the page data using `addItem`. On success format the new/changed page using `viewPage`. On failure ask the user to try again using `editForm`.

Note that the views and models do not directly interact with other components -- the controllers define the interaction between components.

Configuring the application directory

To build any application using WHIFF the top level components must reside in a configured WHIFF application directory. For wwiki the directory is INSTALL/demo/wwiki/wikiroot, configured by the file INSTALL/demo/wwiki/wikiroot/__init__.py which has the following standard content.

from whiff import resolver

__wsgi_directory__ = True

resolver.publish_templates(__path__[0], globals())

This configuration identifies the directory as a directory for serving WSGI applications and also loads any configuration templates in the directory.

The views standing alone

The MVC approach ‡

×
And similar approaches. MVC has a lot of mind-share, but I think there are a lot of ways to organize applications in a disciplined manner.

frequently allows the components of an application to be developed and tested in isolation. ‡

×
So you don't have to treat the whole application as a single intractible ball of goo.

For the wiki application we may develop and examine the view components separately from the other components. For example INSTALL/demo/wwiki/wikiroot/formatPage.whiff, shown below, specifies dummy default values for the external inputs Name, Title, Summary, Text, Note, and TimeStamp.

{{env whiff.content_type: "text/html"/}}

<html>
<head>
<title> {{get-env Name}}DEFAULT_NAME{{/get-env}}: 
	{{get-env Title}}Default title for testing{{/get-env}} </title>
</head>
<body style="width:500px;">

<a href="listPages">return to page list.</a>

<h3> {{get-env Name}}DEFAULT_NAME{{/get-env}}: 
	{{get-env Title}}Default title for testing{{/get-env}} </h3>

<h4>Summary</h4>

<blockquote><b>
	{{get-env Summary}}
	Default summary for testing.  If you see this in production
	something is wrong.
	{{/get-env}}
</b></blockquote>

<h4>Description</h4>
<blockquote>
	{{get-env Text}}
	Default Text for testing.  You should never see this in a "real" page.
	{{/get-env}}
</blockquote>

<center>
<em>[last edit: {{get-env TimeStamp}}00:00:00:00{{/get-env}}]</em><br>
	<a href="editPage?Name={{get-env Name}}DEFAULT_NAME{{/get-env}}">edit this page</a><br>
	<a href="addPage">add new page</a>
</center>

<!-- {{get-env Note}}default test note{{/get-env}} -->

</body>
</html>

In the real intended usage, INSTALL/demo/wwiki/wikiroot/formatPage.whiff should extracts external inputs from the execution environment mapping, for example inserting the Note value from the WSGI environment mapping in place of this get-env directive

{{get-env Note}}default test note{{/get-env}}

but if the environment entry Note is missing the page uses the listed default value default test note instead.

Since all external inputs have defaults we may view the page using no external inputs. To do this we may launch a simple test server for the INSTALL/demo/wwiki/wikiroot. directory.

...$ cd INSTALL/demo/wwiki/
INSTALL/demo/wwiki$ whiff_serve.py wikiroot/
serving whiff root <module 'wikiroot' from './wikiroot/__init__.pyc'>
as http on host localhost port 8888

With the test server running we can view the page with default values by directing the browser to http://localhost:8888/formatPage.

The Note value doesn't show up in the browser because it's inside an HTML comment. ‡

×
Use the browser "view source" feature to find it.

In a similar manner you may examine the other views populated with default/test values in the browser:

View INSTALL/demo/wwiki/wikiroot/editForm.whiff at http://localhost:8888/editForm.
View INSTALL/demo/wwiki/wikiroot/formatPageList.whiff at http://localhost:8888/formatPageList.
View INSTALL/demo/wwiki/wikiroot/pageSummary.whiff at http://localhost:8888/pageSummary. ‡
×
The view formatPageList makes use of the standard repeat middleware and pageSummary to generate a list of page summaries. This tutorial does not go into detail on these pages because they are not central to the MVC discussion.

Adding stubbed models

An MVC architecture makes it easy to provide "stub" implementations for components, such as the model logic. Stubbing can make it simpler to connect the components together before the model logic is completely implemented or configured. ‡

×
It can also make it easier to build unit tests for components without making the unit tests dependent on the implementation of other components.

To allow the model components to contain implementation stubs we make use of a special environment flag STUB and we implement the controllers to provide test functionality when they detect the flag.

For example here is the stub logic for getItem which pretends to retrieve a wiki page, but instead just puts static data into the execution environment for the page argument.

Stubbed getItem

Recall the responsibility of the getItem model component:

Get wiki page data using the name of the item.

We implement getItem as a middleware which retrieves the page data, puts the data into a fresh copy of the WSGI environment dictionary, and then sends the request to another application. The other application is "injected" as the page argument when the getItem middleware is initialized, and this page is responsible for generating the HTML response to the request.

The getItem functionality is implemented using Python code by the file INSTALL/demo/wwiki/wikiroot/getItem.py with the following content

from whiff.middleware import misc
import time
from whiff import gateway
from whiff import resolver
from whiff import whiffenv

class getItem(misc.utility):
    
    def __init__(self, page):
        self.page = page
        
    def __call__(self, env, start_response):
        env = env.copy()
        is_stub = env.get("STUB")
        if is_stub:
            env = self.stub(env)
        else:
            env = self.update(env)
        return self.deliver_page(self.page, env, start_response)
    
    def stub(self, env):
        env["Name"] = "STUB_WIKI_ITEM_NAME"
        env["Title"] = "Stub Title"
        env["Summary"] = "Stub Summary"
        text = "This is just some stub text for testing purposes. Here is a [[Link]] notation."
        env["Text"] = addLinks(text)
        env["TextSource"] = text
        env["Note"] = "Stub Note." 
        env["TimeStamp"] = time.ctime()
        return env

    def update(self, env):
	# OMITTED FOR NOW (SEE BELOW)

def addLinks(text):
    """format [[link]] as an internal wiki reference"""
    L = []
    startSplit = text.split("[[")
    L.append(startSplit[0])
    for chunk in startSplit[1:]:
        endSplit = chunk.split("]]")
        if len(endSplit)<2:
            L.extend(endSplit)
        else:
            name = endSplit[0]
            remainder = endSplit[1:]
            L.append('<a href="viewPage?Name=%s"> [%s] </a>' % (name, name))
            L.extend(remainder)
    return "".join(L)

__middleware__ = getItem

The real functionality of getItem will be implemented in the update method. which will access an external data store and fill in environment entries for the wiki page of interest before delegating the page formatting to the page argument. The fake stub functionality pretends to get the page information, but puts in static values like Stub Title for Title instead. ‡

×
Both the stubbed and the real functionality make use of the addLinks filter which looks in the page text for marks like [[PageLink]] and translates those marks into internal links to other wiki pages.

The code segment

is_stub = env.get("STUB")
if is_stub:
    env = self.stub(env)
else:
    env = self.update(env)

Detects whether the STUB environment parameter is set and switches to stubbed behaviour if it is.

Stubbed addItem

The task of the addItem model middleware was more complex

Change or add wiki page data for a named paged. Validate the data and fail if the data is not complete.

We implement addItem as a middleware which delegates to one of two other applications depending on whether the wiki page data is complete -- if the data is complete the successPage application handles the request, but if the data is not complete the failurePage handles the request. Both the successPage and the failurePage are injected into the addItem on initialization.

The addItem middleware implemented by the Python source file INSTALL/demo/wwiki/wikiroot/addItem.py has even simpler stub logic:

from whiff import gateway
from whiff import resolver
from whiff.middleware import misc
from whiff import whiffenv
import time

class addItem(misc.utility):
    
    def __init__(self, successPage, failurePage):
        self.successPage = successPage
        self.failurePage = failurePage
        
    def __call__(self, env, start_response):
        env = resolver.process_cgi(env, parse_cgi=True)
        name = whiffenv.cgiGet(env, "Name", "").strip()
        title = whiffenv.cgiGet(env, "Title", "").strip()
        summary = whiffenv.cgiGet(env, "Summary", "").strip()
        text = whiffenv.cgiGet(env, "Text", "").strip()
        note = whiffenv.cgiGet(env, "Note", "").strip()
        error_message = ""
        if not (name and title and summary and text):
            error_message = "please provide all required data: name, title, summary and text"
        is_stub = env.get("STUB")
        timeStamp = time.ctime()
        if not error_message:
            if is_stub:
                pass # stub: don't actually do the update
            else:
                self.insertItem(env, name, title, summary, text, note, timeStamp)
            return self.deliver_page(self.successPage, env, start_response)
        else:
            env["error_message"] = error_message
            return self.deliver_page(self.failurePage, env, start_response)

    def insertItem(self, env, name, title, summary, text, note, timeStamp):
	# OMITTED FOR NOW (SEE BELOW)
    
__middleware__ = addItem

In this case in STUB mode the middleware simply doesn't call the insertItem action which implements the logic for inserting or changing a page. ‡

×
Much of the remaining logic for addItem is not stubbed out: the middleware finds the submitted form data; checks that the form data is complete; if the data is complete delegates to the successPage; if the data is not complete delegates to the failurePage.

Stubbed getItemList

The getItemList model component has the following responsibility

Get summary information for all wiki pages.

The getItemList middleware is similar to the getItem middleware -- it puts some information in the execution environment and delegates the request to the injected page application. It is implemented in the Python source file INSTALL/demo/wwiki/wikiroot/getItemList.py which has the following content

from whiff import gateway
from whiff.middleware import misc
import time

class getItemList(misc.utility):
    
    def __init__(self, page):
        self.page = page
        
    def __call__(self, env, start_response):
        env = env.copy()
        is_stub = env.get("STUB")
        if is_stub:
            env = self.stub(env)
        else:
            env = self.update(env)
        return self.deliver_page(self.page, env, start_response)
    
    def stub(self, env):
        env["pageList"] = [
            [ "FIRST_PAGE_NAME_STUB", "Stub Title For First Page" ],
            [ "SECOND_PAGE_NAME_STUB", "Stub Title For Second Page" ],
            [ "THIRD_PAGE_NAME_STUB", "Stub Title For Third Page" ]
            ]
        return env

    def update(self, env):
	# OMITTED FOR NOW (SEE BELOW)

__middleware__ = getItemList

Here the STUB implementation modifies the environment by adding a literal list of pairs named pageList summarizing imaginary wiki pages, This fake environment is then passed on to the page application.

Injecting views into models using controllers

The stub model implementations are middlewares -- they modify the execution environment and then delegate the generation of the page response to other applications. The other applications are injected as arguments to the model component by the controllers. ‡

×
The method of using third party controller to inject a view into a model instance is sometimes goes by the confusing name of dependency injection.

For example the controller configuration INSTALL/demo/wwiki/wikiroot/formatPageList.whiff invokes getItemList to get the list of wiki page summaries injecting formatPageList to format the resulting summaries list. The source for the configuration is as follows:

{{include "getItemList"}}
        {{include "formatPageList"/}}
{{/include}}

Here the interpretation of the configuration creates an instance of the getItemList middleware using the application interpretation of the formatPageList configuration as the page argument.

This usage is so trivial that it begs the question: why is such a trivial connection useful? The reason for using a controller to connect getItemList to its formatting application is to allow the getItemList component to potentially work with other formatters. For example we could imagine another configuration which formats an Adobe Portable Document Format (PDF) document containing the page list instead of formatting an html page:

{{include "getItemList"}}
        {{include "formatPageListPDF"/}}
{{/include}}

By separating the formatter from the model logic the model logic can be used for any number of other purposes.

Launching a test server with stubbed model functionality

To view the output of the controller formatPageList described above using the stubbed getItemList functionality we must run a server that configures the application directory wikiroot with the environment flag STUB turned on. The following Python script INSTALL/demo/wwiki/stubWiki.py launches a server for the directory in stubbed mode.

from whiff import resolver
import wikiroot

testapp = resolver.moduleRootApplication("/", wikiroot,
                                         environment={"STUB": True},
                                         )

if __name__=="__main__":
    import wsgiref.simple_server
    print "serving wwiki in STUB mode at 8888"
    print "start page at http://localhost:8888/listPages"
    srv = wsgiref.simple_server.make_server('localhost', 8888, testapp)
    srv.serve_forever()

The key line of this otherwise standard configuration is the resolver.moduleRootApplication argument

environment={"STUB": True}

which sets the STUB flag in the environment for all applications launched by the server.

We launch the stubbed server by running the script

INSTALL/demo/wwiki$ python stubWiki.py
serving wwiki in STUB mode at 8888
start page at http://localhost:8888/listPages

Then when we point a browser at http://localhost:8888/listPages we see:

More controllers

viewPage

If we click on the one of the links to the fake wiki pages above the link requests a response from the viewPage?Name=NAME_OF_WIKI_PAGE URL implemented by the INSTALL/demo/wwiki/wikiroot/viewPage.whiff controller configuration:

{{include "getItem"}}
	{{include "formatPage"/}}
{{/include}}

The configuration above essentially has getItem find the data associated with the wiki page and then pass the data to formatPage to generate the HTML response. In STUB mode the getItem model component ignores the Name parameter and always return the same static stub data.

In stub mode the only thing that changes is the timestamp.

editPage

If we click on the edit this page link above the browser requests a response from the URL editPage?Name=STUB_WIKI_ITEM_NAME which is implemented by the INSTALL/demo/wwiki/wikiroot/editPage.whiff controller configuration:

{{include "getItem"}}
	{{include "editForm"/}}
{{/include}}

The configuration above is very similar to INSTALL/demo/wwiki/wikiroot/viewPage.whiff except that the data from getItem is sent to editForm to present an edit form to the user.

The editForm view is implemented by the file INSTALL/demo/wwiki/wikiroot/editForm.whiff with content

{{env whiff.content_type: "text/html"/}}

<html>
<head>
<title> Edit Wiki Entry </title>
</head>
<body style="width:500px;">

<b><em>{{get-env error_message}}{{/get-env}}</em></b>

<h3> Edit Wiki Entry </h3>

<form action="changePage" method="post">
	Name: <input name="Name" value="{{get-env Name}}WIKI_ITEM_NAME{{/get-env}}"> 
		(required)<br>
	Title: <input name="Title" size="50" 
		value="{{get-env Title}}TEST DEFAULT Title for Wiki Item Name{{/get-env}}"/> 
		(required)<br>
	Summary: (required) <br>
		<textarea cols="50" rows="3" name="Summary">{{get-env Summary}}
		This TEST DEFAULT text summarizes the entry about Wiki Item Name.
		{{/get-env}}</textarea> <br>
	Text: (required) <br>
		<textarea cols="50" rows="10" name="Text">{{get-env TextSource}}
		This TEST DEFAULT text explains the concept "Wiki Item Name". link: [[WIKI_ITEM_NAME]].
		{{/get-env}}</textarea> <br>
	Note: <input name="Note" size="50" value=""><br>
	<input type="submit" name="save" value="save">
</form>
</body>
</html>

In the editPage configuration the getItem middleware overrides all the default input values provided by the editForm view. In STUB mode the resulting HTML looks like this in a browser.

changePage

If we then click on the save button the browser submits the form contents to the URL changePage implemented by the configuration template INSTALL/demo/wwiki/wikiroot/changePage.whiff.

{{include "addItem"}}
	{{use-include successPage "viewPage"/}}
	{{using failurePage}}
		{{cgi-default Name}}{{/cgi-default}}
		{{cgi-default Title}}{{/cgi-default}}
		{{cgi-default Summary}}{{/cgi-default}}
		{{cgi-default Text}}{{/cgi-default}}
		{{include "editForm"}}
			{{set-id Name}}"{{get-cgi Name/}}"{{/set-id}}
			{{set-id Title}}"{{get-cgi Title/}}"{{/set-id}}
			{{set-id Summary}}"{{get-cgi Summary/}}"{{/set-id}}
			{{set-id TextSource}}"{{get-cgi Text/}}"{{/set-id}}
		{{/include}}
	{{/using}}
{{/include}}

This configuration uses the addItem middleware to attempt to add a new wiki item or change an existing wiki item. If the add operation succeeds the response to the request will be generated by the successPage which is bound to the viewPage application and viewPage will present the successfully changed wiki item.

However if the add operation fails the response to the form submission will be generated by the failurePage which invokes the editForm application to let the user fill in the form again. The cgi-default directives make sure that any missing cgi variables are assigned default empty string values. The set-id directives map any values submitted in the form into the appropriate environment entries so the user doesn't have to type all the values again. ‡

×
Note that

{{set-id Name}}"{{get-cgi Name/}}"{{/set-id}}

puts quotes around the name value. This is because the set-id directive requires a JSON formatted value in order to allow set-id to install more complex data structures into the environment when required.

One way to get to see the failurePage in action is to refuse to provide a title when you submit the edit form.

The message please provide all required data: name, title, summary and text was installed in the application environment by the addItem middleware as shown in the listing above:

if not (name and title and summary and text):
    error_message = "please provide all required data: name, title, summary and text"
...
if not error_message:
    ....
else:
    env["error_message"] = error_message
    return self.deliver_page(self.failurePage, env, start_response)

The editForm view pulled the error message out of the environment at the line

<b><em>{{get-env error_message}}{{/get-env}}</em></b>

addPage

For completeness here is the implementation of the addPage controller component implemented by the configuration template INSTALL/demo/wwiki/wikiroot/addPage.whiff.

{{env Name: "",
        Title: "",
        Summary: "",
        TextSource: ""
/}}
{{include "editForm"/}}

This template delegates the response to the editForm view after forcing all of the input parameters for editForm to have empty string values.

Configuring a simple file based storage resource

In order to allow the wiki application to store and retrieve pages we define and deploy a WHIFF resource for the application root directory. For the purposes of this tutorial we will keep the semantics of the resource extremely simple

All WHIFF resources are identified by resource path lists which the resource manager uses to locate the resource. For the wiki storage let's use the very simple path list:

["storage"]

This resource will represent the storage for the entire wiki as a single object -- a Python dictionary. ‡

×
Using a single big dictionary to represent the wiki is very convenient from a programming point of view, but it might become unmanageable if the wiki becomes large. For the purposes of this tutorial we are not attempting to implement a large wiki with more than a few thousand entries.

If we were to implement a really large wiki many other components would require revisions and enhancements. Modifying the storage to support millions of entries is not difficult, but such details are not in line with the main purpose of this tutorial.

All WHIFF resources have associated values. In the present case we will implement the storage resource as a dictionary mapping wiki page names to dictionaries containing wiki page descriptions. For example a wiki with three entries might have the resource value:

{
	"alpha": {	"Name": "alpha",
			"Title": "Alpha",
			"Summary": "greek letter",
			"Text": "Alpha is the first letter of the greek alphabet",
			"Note": "typo: 'geek' should read 'greek'",
			"Timestamp": "Thu Apr 30 16:39:24 EDT 2009"
		},
	"beta": {	"Name": "beta",
			"Title": "Beta",
			"Summary": "greek letter",
			"Text": "Beta is the second letter of the greek alphabet",
			"Note": "typo: 'bertha' should read 'beta'",
			"Timestamp": "Thu Apr 30 17:39:24 EDT 2009"
		},
	"gamma": {	"Name": "gamma",
			"Title": "Gamma",
			"Summary": "greek letter",
			"Text": "Gamma is the third letter of the greek alphabet",
			"Note": "typo: 'gramma' should read 'gamma'",
			"Timestamp": "Thu Apr 30 18:39:24 EDT 2009"
		},
}

We will provide two implementations for storing and retrieving the resource. The first will be quick-and-dirty using a file to store the value, and the second will be more robust, using a mySql database to store the value.

To store the wiki dictionary to a file we implement a simple resource finder that uses the marshal module to serialize and deserialize the data into a file. This simple resource finder is implemented by the Python code file INSTALL/demo/wwiki/wikiStore.py with content

import marshal
import os

class wikiStorage:
    
    def __init__(self, filepath, value=None):
        self.filepath = filepath
        self.value = value
        
    def localize(self, env):
        "return a connection between an HTTP request and the storage resource"
        # read in and cache the any data from the file for this request
        value = None
        if os.path.exists(self.filepath):
            f = file(self.filepath, "rb")
            value = marshal.load(f)
        # return a connected storage object
        return wikiStorage(self.filepath, value)
    
    def get(self, pathlist):
        return self.value

    def put(self, pathlist, value):
        self.value = value
        f = file(self.filepath, "wb")
        marshal.dump(value, f)
        f.close()

The wikiStorage.__init__ method records the path to the file which will store the wiki dictionary data. The localize method reads and caches the dictionary for a given HTTP request which needs the resource. The get operation returns the cached value, and the put operation records a new cached value as well as marshal-dumping the value to the file.

Unstubbing the model components to use the storage resource

To "unstub" the model components we add logic to get and put the storage resource as appropriate to purpose of the model component.

getItem unstubbed

The getItem middleware gets information about a wiki entry. The earlier stub listing for getItem left out the logic for extracting the data from storage. The abbreviated INSTALL/demo/wwiki/wikiroot/getItem.py listing below fills in the method update which gets the page information out of the wiki dictionary representation. ‡

×
Code segments listed previously are replaced by .... ellipses.

from whiff import resolver
from whiff import gateway
from whiff.middleware import misc
....

class getItem(misc.utility):
    ....

    def update(self, env):
        env = resolver.process_cgi(env, parse_cgi=True)
        name = whiffenv.cgiGet(env, "Name", "").strip()
        if not name:
            raise resolver.ModuleRootResolutionException, "cannot find wiki item with empty name"
        storage = gateway.getResource(env, ["storage"])
        if storage is None:
            storage = {}
        if not storage.has_key(name):
            raise resolver.ModuleRootResolutionException, "no page found with this name: "+repr(name)
        data = storage[name]
        env["Name"] = data["Name"]
        env["Title"] = data["Title"]
        env["Summary"] = data["Summary"]
        text = data["Text"]
        env["Text"] = addLinks(text)
        env["TextSource"] = text
        env["Note"] = data["Note"]
        env["TimeStamp"] = time.ctime()
        return env
....

__middleware__ = getItem

Here the line

name = whiffenv.cgiGet(env, "Name", "").strip()

Gets the name of the page to extract from the request CGI input parameters.

The line

storage = gateway.getResource(env, ["storage"])

gets the wiki dictionary resource from the WHIFF resource manager.

The remainder of the update implementation finds the description for the requested page and unpacks the description into the WSGI environment dictionary.

addItem unstubbed

To unstub the addItem middleware listed previously we fill in the definition for the omitted indertItem operation.

from whiff import gateway
from whiff.middleware import misc
....

class addItem(misc.utility):
    ....    

    def insertItem(self, env, name, title, summary, text, note, timeStamp):
        data = {}
        data["Name"] = name
        data["Title"] = title
        data["Summary"] = summary
        data["Text"] = text
        data["Note"] = note
        data["TimeStamp"] = timeStamp
        storage = gateway.getResource(env, ["storage"])
        if storage is None:
            storage = {}
        storage[name] = data
        gateway.putResource(env, ["storage"], storage)
    
__middleware__ = addItem

Here the line

gateway.putResource(env, ["storage"], storage)

directs the WHIFF resource manager to record a new value for the storage wiki representation.

getItemList unstubbed

The unstubbing of getItemList is very similar to the unstubbing of getItem

from whiff.middleware import misc
from whiff import gateway
....

class getItemList(misc.utility):
    ....

    def update(self, env):
        storage = gateway.getResource(env, ["storage"])
        if storage is None:
            # no entries
            env["pageList"] = []
            return env
        entries = storage.values()
        namesAndSummaries = [ [e["Name"], e["Summary"]] for e in entries ]
        namesAndSummaries.sort()
        env["pageList"] = namesAndSummaries
        return env

__middleware__ = getItemList

Running the server unstubbed

To serve the wiki interface using the unstubbed model components we create a Python script which deploys a server to use the wikiStore resource finder, implemented by the Python source file INSTALL/demo/wwiki/fileWiki.py with content

from whiff import resolver
from whiff.middleware import displayTraceback
import wikiroot
import wikiStore

# create an application to serve the wikiroot directory
testapp = resolver.moduleRootApplication("/", wikiroot)

# define a resource finder which stores wiki data to the file "/tmp/wikiData.dat"
storage = wikiStore.wikiStorage("/tmp/wikiData.dat")

# register the resource finder with the name "storage"
testapp.registerResourceFinder("storage", finder=storage)

if __name__=="__main__":
    import wsgiref.simple_server
    print "serving wsgi chat test at 8888"
    print "start page at http://localhost:8888/listPages"
    srv = wsgiref.simple_server.make_server('localhost', 8888, testapp)
    srv.serve_forever()

Running the script launches a test server using the file based storage resource.

INSTALL/demo/wwiki$ python fileWiki.py 
serving wsgi chat test at 8888
start page at http://localhost:8888/listPages

The pages served by the server look similar to the stubbed pages, but pages can be created, saved, and modified.

After adding some pages we see the following list of entries

Following the "beta" link we see

Following the "edit this page" link we see

... And so forth

Deploying the wiki as a CGI script

The wwiki application directory may now be installed in any web server configuration supporting the WSGI interface. For example to add the application running as a CGI script under an Apache server, ‡

×
In my case on a Macintosh laptop running a Unix variant.

I create the CGI script file /usr/local/apache2/cgi-bin/wwiki.cgi with content

#!/PATH/TO/python

import sys
from  whiff import resolver
import wsgiref.handlers

# add the "wwiki" root directory to the python path
sys.path.append("/Users/Aaron/whiff/demo/wwiki")
import wikiroot
import wikiStore

# set up the root directory application.
application = resolver.moduleRootApplication("/cgi-bin/wwiki.cgi", wikiroot,)

# configure the storage resource for the root directory application.
storage = wikiStore.wikiStorage("/tmp/wikiData.dat")
application.registerResourceFinder("storage", finder=storage)

# serve a CGI request using the directory
wsgiref.handlers.CGIHandler().run(application)

It is important to notice that the root application configuration needs to be told it's server relative URL location "/cgi-bin/wwiki.cgi" -- apart from that detail this configuration is very similar to the test server configuration above.

I also need to make sure that the CGI script is executable and that the file used for archiving the wiki is readable and writable by the CGI processes.

/usr/local/apache2/cgi-bin$ chmod a+x wwiki.cgi
/usr/local/apache2/cgi-bin$ chmod a+rw /tmp/wikiData.dat

Now we can see the wiki page list served by the Apache server at http://aaron.oirt.rutgers.edu/cgi-bin/wwiki.cgi/listPages -- please have a look!

Configuring a mySql database storage resource

In order to make the storage resource more robust ‡

×
The marshal file storage implementation leaves open the possibility that two updates will "collide" and one of the updates will be lost (among other possible issues). A MySql storage implementation may prevent collisions of this sort, if the engine is configured properly, as well as provide other advantages.

we provide an alternate implementation for the storage resource implemented in the Python module source file INSTALL/demo/wwiki/mySqlWikiStore.py with content

import MySQLdb

# sql to create the tables for the application

TABLE_CREATES = [
    """
    create table wwikientries (
        name varchar(100),
        title varchar(100),
        note varchar(200),
        summary varchar(200),
        timestamp char(20),
        the_text text
    )
    """,
    """
    create table wwikilogs (
        name varchar(100),
        title varchar(100),
        note varchar(200),
        summary varchar(200),
        timestamp char(20),
        the_text text
    )
    """,
    ]

# sql modification operations

INSERT_ENTRY = """
    insert into wwikientries
    (name, title, note, summary, timestamp, the_text)
    values (%s, %s, %s, %s, %s, %s)
"""

INSERT_LOG = """
    insert into wwikilogs
    (name, title, note, summary, timestamp, the_text)
    values (%s, %s, %s, %s, %s, %s)
"""

DELETE_ENTRY = """
    delete from wwikientries where name=%s
"""

# sql query operations

TEST_QUERY = "select * from wwikientries limit 1"

GET_ENTRIES = """
    select name, title, note, summary, timestamp, the_text
    from wwikilogs
    order by name
"""

class wikiStorage:
    
    def __init__(self, host, user, passwd, db):
        self.host = host
        self.user = user
        self.passwd = passwd
        self.db = db
        self.connection = None
        self.entriesDictionary = None
                
    def localize(self, env):
        "return a connection between an HTTP request and the storage resource"
        result = wikiStorage(self.host, self.user, self.passwd, self.db)
        # make sure the database is set up
        result.initTables()
        # get and cache entries
        result.getEntries()
        return result
    
    def get(self, pathlist):
        return self.entriesDictionary.copy()

    def put(self, pathlist, value):
        oldvalue = self.entriesDictionary
        for (name, entry) in value.items():
            if oldvalue.get(name)!=entry:
                self.addEntry(entry)
        # refresh cached data from the database
        self.getEntries()

    def connect(self):
        "get cached connection to the database for the current request, or make a connection and cache it and return it"
        if self.connection is None:
            self.connection = MySQLdb.connect(host=self.host, 
		user=self.user, passwd=self.passwd, db=self.db)
        return self.connection

    def initTables(self):
        "make sure the wwiki tables exist, or create them if not"
        cursor = self.connect().cursor()
        try:
            cursor.execute(TEST_QUERY)
        except:
            for statement in TABLE_CREATES:
                cursor.execute(statement)

    def getEntries(self):
        cursor = self.connect().cursor()
        result = {}
        n = cursor.execute(GET_ENTRIES)
        for (name, title, note, summary, timestamp, text) in cursor.fetchall():
            D = {
                "Name": name,
                "Title": title,
                "Note": note,
                "Summary": summary,
                "TimeStamp": timestamp,
                "Text": text,
                }
            result[name] = D
        self.entriesDictionary = result
        return result
    
    def addEntry(self, entry):
        name = entry["Name"]
        title = entry["Title"]
        note = entry["Note"]
        summary = entry["Summary"]
        timestamp = entry["TimeStamp"]
        text = entry["Text"]
        conn = self.connect()
        cursor = conn.cursor()
        cursor.execute(DELETE_ENTRY, (name,))
        cursor.execute(INSERT_ENTRY, (name, title, note, summary, timestamp, text))
        cursor.execute(INSERT_LOG, (name, title, note, summary, timestamp, text))
        conn.commit()

This SQL based storage resource uses two tables to store information about wiki entries: the wwikientries table stores current values for named entries and the wwikilogs table stores current entries and any old entries which have been superseded.

The __init__ method stores the database connection information, but does not interact with the database directly.

The localize method allocates a storage object for a web request and caches a connection to the database dedicated for that request. It also tests to make sure the database tables exists, and creates them if they do not exist. After verifying that the table exists the localize method reads and caches the wiki data as a dictionary. ‡

×
As discussed previously, this implementation is designed for wikis with up to several thousand entries. A larger wiki might be better implemented using a more optimized approach which gets only the part of information needed for a given page, rather than the all wiki entries at once.

The get method returns the cached wiki dictionary previously read and cached by localize.

The put method compares the new value against the cached value and inserts a new entry or replaces an old entry for any new data not found in the old data. ‡

×
The other methods not described support the localize and put methods.

Configuring a root application with the mySql database storage resource

The python script INSTALL/demo/wwiki/mySqlWiki.py configures a WHIFF root application to use the mySql based storage resource.

from whiff import resolver
from whiff.middleware import displayTraceback
import wikiroot
import mySqlWikiStore

testapp = resolver.moduleRootApplication("/", wikiroot)

storage = mySqlWikiStore.wikiStorage("host", "user", "password", "database")

testapp.registerResourceFinder("storage", finder=storage)

if __name__=="__main__":
    import wsgiref.simple_server
    print "serving wsgi chat test at 8888"
    print "start page at http://localhost:8888/listPages"
    srv = wsgiref.simple_server.make_server('localhost', 8888, testapp)
    srv.serve_forever()

This script is exactly the same as fileWiki.py except for the line

storage = mySqlWikiStore.wikiStorage("host", "user", "password", "database")

which deploys the mySql based resource configured with database connection information. To run the application using a test server, edit the database connection information to have valid parameters and run the script from the command line.

Note that the logic of the application works with both resource implementations without modification: only the root configuration changes.

Summary

This discussion has illustrated a lot of techniques for web design and development.

We sketched a MVC design for the wwiki application and identified the views, models, and controllers for the application and how they connect together.
We built the views for the application in "stand alone" mode.
We added stub implementations for the model components without fully implementing the data storage methods.
We created controllers that connected the (stub) model components to the view components.
We tested the MVC infrastructure against the model stubs using a test server deployed in STUB mode.
We implemented unstubbed functionality for the model components defined against a WHIFF storage resource.
We implemented the WHIFF storage resource using two different strategies.
We deployed servers using the different storage resource implementations without changing the application logic

<previous<
A resume wizard

^up^
Tutorial

>next>
Drop down menus for navigation

0 comments.

Care to comment?

name: (required)
- email (not published):
comment: (required)

<< security number? >>