Loud Baking

One of the things I hate the most when developing an application is counting on 3rd parties to deliver or reply on time. Be it freelancers, API developers or sometimes your own partners (banks, lawyers, etc.). In this case, it’s the API developers. A couple days ago, I emailed Google for some help on finding ‘My Client Center’ link under the ‘Account’ tab in my Adwords account. After waiting 4 days, when I finally received an email from their support team, I was excited, but not for long. Even though I had sent a specific request, explaining that I have already received my developer token but couldn’t log to my developer account, all I got back is a quick-reply style of email with the instructions on how to apply for a developer token! Looks like when you’re Google you can permit yourself everything from late replies to stupid staff.

Anyhow, this obviously affects the development of a vendor class I wanted to finish, but nothing that could stop me from moving on and pushing it back a couple of weeks. In the worst case scenario, if they just tell me that my developer token is not valid anymore (inactivity for a long period is the only reason I could think of) and since they don’t give out developer tokens anymore, I will find myself obliged to completely scrap that feature from the application. It sucks, I know, but that’s the price to pay when some of your features depend on 3rd parties I guess.

On the positive side of things, everything has been going according to the initial timeline with a couple of improvements to be made on the design and finalizing the teaser page for early invitations.

During the past couple of years, Ruby on Rails has been getting massive attention and I never really got the time to dive into it - learned a couple things here and there, but haven’t really developed any application with it. I liked it for the same reasons that everyone else does: it’s a rapid development framework using the MVC pattern and ActiveRecord.

Learning a new language (in this case 2, Ruby and Rails) is definitely exciting, but having to do that while developing a pretty advanced web application that needs to go live in less than 2 months, raises the bar pretty high.

So I opted for goold ol’ PHP. I have built all kinds of little scripts, hacked many open-source projects (as in modified them) and released 4 of my own turnkey solutions all based on PHP but never had I used an open-source framework for that. This time, things will change. I said in the first post something about CakePHP which is the one I selected for this application and probably more to come.

As Jonathan Snook says it so well:

I almost fear putting this kind of post together as it’s bound to pull the fanatics (in the negative sense of the word) out of the woodworks.

So instead of giving a comparison of the available PHP MVC frameworks, I will instead list what I was looking for and believed CakePHP would cover. Here it is:

• Open Source / Free License
• KISS
• OO and MVC
• ORM / Active Record
• Security
• Ajax interoperability
• Good controller structure
• Good helpers available
• Scaffolding

I did try a couple before making my final decision, but here are the resources that helped me make up my mind:

- Taking a look at ten different PHP frameworks, by Dennis Pallett
- CodeIgniter vs. CakePHP, by Jonathan Snook
- New Year’s Benchmarks and A Bit About Benchmarks, by Paul M. Jones
- Glue vs. Full Stack and More framework fun, by Chris Hartjes
- Comparing Frameworks, by Tim Bray

Been pretty busy lately with csstester.com which is soon to be launched, finalizing stuff, etc.

I know, I know - I should only be concentrating on one application at a time, but that ain’t my kind. Am not smarter or anything, I just get bored too fast from working on the same thing project every single day, all day.

Anyhow, back to the reason for this post, Google AdWords API and the awful experience I had so far.

Back in early 2006, I had applied and received my developer token for the API (which Google stopped distributing as of October 2006). Developer token, email account to which the token was emailed and password in hand, I head to the login page. To my big surprise, Google replies back with a message along the lines of:

This account does not exist

Ok, so you emailed me a developer token and an API key in a separate email, to that exact same email address I am using to access my account, but you don’t believe I have an account. Great!

Since my developer email access is not the main admin access to the account, I log with my other email and invite the email Google had sent me the developer token to. A couple steps after, I am logged into my account using the developer credentials.

According to AdWords’ welcome email, I am supposed to find a ‘My Client Center’ link under the ‘Account’ tab.  But nothing there.

I go looking for help, reading pages and pages of information I could have skipped for the time being, but couldn’t find any kind of help about the issue I was having.

Last resort: email Google.

It’s been nearly 12 hours, and still no reply. One would have believed that between their  free meals or exercising, someone could have a look and just fix that in my account.

Waiting…

I discussed the other day about writing the use cases. I also mentioned that I was going to write them all in XML so they could be easily parsed later on.

As you might have noticed by looking at only 3 use cases, it gets pretty long and hard to read in raw format, so first code had to be a parser for that.

Solution: Clean XML To Array by Ivan Enderlin, found on PHPClasses.

All you need is to download the lib.xml.php and create the a new file with the little code shown below.

<?php
include('lib.xml.php');
$xml = new Xml;
$out = $xml->parse('file.xml', 'FILE');
echo '<pre>'.print_r($out).'</pre>';
?>

All that was left to do was to parse it in an easy to read format. This will do for now as I don’t want to spend too much time on it.

<?php
include('lib.xml.php');
$xml = new Xml;
$out = $xml->parse('usercase.xml', 'FILE');
//echo '<pre>'.print_r($out).'</pre>';exit();

foreach ($out as $usecases) {
	for ($i=0; $i<count($usecases); $i++) {
		$usecase = $usecases[$i];

		echo '<div class="usecase">';
		echo '<h3>'.$usecase[name].': '.$usecase[description].'</h3>';
		echo '<p class="overview">';
		echo '<strong>Sitting:</strong> '.$usecase[sitting].'<br />';
		echo '<strong>Primary Actor:</strong> '.$usecase[primaryactor].'<br />';
		echo '<strong>Scope:</strong> '.$usecase[scope].'<br />';
		echo '<strong>Level:</strong> '.$usecase[level].'<br />';
		echo '<strong>Minimal Guarantee:</strong> '.$usecase[minimalguarantee].'<br />';
		echo '<strong>Success Guarantee:</strong> '.$usecase[successguarantee].'<br />';
		echo '</p>';

		echo '<h4>Stakeholders</h4><p class="stakeholders">';
		foreach($usecase[stakeholders][stakeholder] as $id => $stakeHolder){
			echo '<strong>'.$stakeHolder.':</strong> '.$usecase[stakeholders][interest][$id].'<br />';
		}
		echo '</p>';

		echo '<h4>Scenario</h4><p class="scenario">';
		if(is_array($usecase[scenario][step])){
			foreach($usecase[scenario][step] as $id => $step) echo $id.'. '.$step.'<br />';
		} else {
			echo $usecase[scenario][step];
		}
		echo '</p>';

		echo '<h4>Extensions</h4><p class="extensions">';
		foreach($usecase[extensions][extension] as $id => $extension){
			echo '<strong>Extends:</strong> '.$extension['extends'].' ('.$usecase[scenario][step][$extension['extends']].')<br />';
			echo $extension['xcase'].'<br />';
			echo '<strong>Steps:</strong><ol>';
			if(is_array($extension['step'])){
				foreach($extension['step'] as $id => $step) echo $id.'. '.$step.'<br />';
			} else {
				echo $extension['step'];
			}
			echo '</ol>';
		}
		echo '</p>';
	}
}

?>

If you are thinking that by using this software your code will be secure, continue reading.

To make a story short, I had in my hands a piece of code that I wanted to study more since I liked it. Not to my surprise, the code was all encrypted. Here is a little piece of it:

if(!function_exists('findsysfolder')){function findsysfolder($fld){$fld1=dirname($fld);$fld=$fld1.'/scopbin';clearstatcache();if(!is_dir($fld))return findsysfolder($fld1);else return $fld;}}require_once(findsysfolder(__FILE__).'/911006.php');

Being a developer that sells scripts, I totally understand the reasons behind encrypting your code but when you do it, do it right.

After digging for less than a minute in the files, I realized that all the decryption functions where there, just written in an awful way, to make it seem all encrypted. Here’s an example:

function A4540acdeed38d4cd9084ade1739498($x897356954c2cd3d41b221e3f24f99bba,$x276e79316561733d64abdf00f8e8ae48){return $Xew6e79316561733d64abdf00f8e8ae48;}

You guessed it right. This function does nothing, there is no point of having it. The file has like 6 of those in the most stupid attempt to lure someone trying to decrypt the code.

Only one function in all that file is necessary and all it requires is to make it more readable, changing the silly function and parameters names to something better than a list of characters.

function y0666f0acdeed38d4cd9084ade1739498($x897356954c2cd3d41b221e3f24f99bba,$x276e79316561733d64abdf00f8e8ae48){$x0b43c25ccf2340e23492d4d3141479dc='';$x71510c08e23d2083eda280afa650b045=0;$x16754c94f2e48aae0d6f34280507be58=strlen($x897356954c2cd3d41b221e3f24f99bba);$x7a86c157ee9713c34fbd7a1ee40f0c5a=hexdec('&H'.substr($x276e79316561733d64abdf00f8e8ae48,0,2));for($x1b90e1035d4d268e0d8b1377f3dc85a2=2;$x1b90e1035d4d268e0d8b1377f3dc85a2<=$x7a86c157ee9713c34fbd7a1ee40f0c5a)$xab6389e47b1edcf1a5267d9cfb513ce5=255+$xab6389e47b1edcf1a5267d9cfb513ce5-$x7a86c157ee9713c34fbd7a1ee40f0c5a;else $xab6389e47b1edcf1a5267d9cfb513ce5=$xab6389e47b1edcf1a5267d9cfb513ce5-$x7a86c157ee9713c34fbd7a1ee40f0c5a;$x0b43c25ccf2340e23492d4d3141479dc=$x0b43c25ccf2340e23492d4d3141479dc.chr($xab6389e47b1edcf1a5267d9cfb513ce5);$x7a86c157ee9713c34fbd7a1ee40f0c5a=$xe594cc261a3b25a9c99ec79da9c91ba5;} return $x0b43c25ccf2340e23492d4d3141479dc;}

10 minutes later, I had the full source code for each file I was interested in.

Bottom line is, if you want to protect your work, spend a bit more than $40 (price of SourceCop when writing this post), because those are a waste and you are better of not spending them then.

I’ve been really vague about the app we’re building here but that’s because I know it’s something that many in the affiliate marketing industry are trying to do right now and I prefer keeping everything like that until pre-launch.

Since 90% of what it does is data tracking and analysis from hundreds of sites. You can imagine now all kinds of settings and tables involved, all the database complexity. To start off, I identified groups based on similarities found at those sites. I then eliminated all the parts that could be just cloned and slightly modified to be added later on. All I had left was the strict minimum for the app to function and be able to run an example of each use case I had already written.

Time to design the database, identify the relations, etc. Here’s a scaled down version of what I came up with after an hour on PHPDesigner DBDesigner 4. I am sure certain things will get added or modified as we go but I believe this is a very solid start.

The new app will be running on a VDS from Myriad Network for the moment. To be more precise, I find that their $56.95 per month package gives enough room to test the waters without any big initial investment. Myriad runs CentOS and to be honest, we ran into a couple of problems installing what was supposed to be very easy.

Here is a quick documentation of how we finally got it to run.

IMPORTANT: The following steps have been tested and worked for us. In no way do I guarantee they will work for every CentOS configuration and strongly suggest you do things carefully and read more documentation on the Trac and Subversion sites.

Installing Trac

• Make sure you have at peast Python 2.4 (otherwise, get ready to face some serious issues)
• Download the latest Trac version - here
• Download ClearSilver - here
• Download Swig - here
• Download SQLite - here
• Download PySQLite - here
• Install Trac
  • Extract files
  • Change to extracted files’ directory
  • Run the following command: python setup.py install
• Install ClearSilver, Swig, SQLite, PySQLite. For each:
  • Extract files
  • Change to extracted files’ directory
  • Run: ./configure
  • Run: male && make install
• Move svnlib to python libraries directory: cp -r /path/to/svnlib /path/to/python2.x/site-packages/
• Test svn bindings from Python CLI
  • Run: python
  • Run: import svn core
  • If everything is ok, you shouldn’t get any errors

Installing Subversion

• Download the latest version of Subversion - here
• Download the latest deps file - here
• Extract subversion
• Extract deps
• Copy/move extracted deps folder content into Subversion root directory
• Run: ./configure –without-neon
• Run: make && make swig-py && make install

Creating a repository

• Run: svnadmin create /path/to/repo
• Change to /path/to/repo/conf and edit svnserve.conf, disable authz-db
• To start SVN as a standalone server, run: svnserve -d -r /path/to/repo
• You can optionally add –listen-port PORT in the command arguments

Setup Trac for repository

• Run: trac-admin path/to/trac-env-root initenv
  • Select a project name
  • Enter path to your subversion repository
• Run: tracd -d -p [PORT] /path/to/trac-env/root /path/to/another-trac-en-root

Subversion cheatsheets:

• http://ariejan.net/svncheatsheet/
• http://www.cs.put.poznan.pl/csobaniec/Papers/svn-refcard.pdf

Notes: Python doesn’t send out any stderr to the system, which in turn makes it hard to debug. Instead, run tracd in interactive mode instead of the daemon mode by simply taking out the ‘-d’ from the tracd -d -p [PORT] command.

To my big surprise, I saw the domain already listed on Google when it’s barely been a month I bought it and it was never used before, go figure!

Some people would do anything to get their domain directly listed on Google without having to go by the sandbox, I know. But what if, because it is now listed and still in development, the search bots fall on a 404 or other kind of errors? You’re right, you start getting penalized.

Now, since the domain is in Google and I better be cautious not to start getting penalized for it, I just created the magic robots.txt file to disallow all crawling starting from the root directory. It’s so damn obvious, but who knows:


User-agent: *
Disallow: *


I could have also made some content pages with the keywords I want the domain to rank for but I don’t have time for that now, maybe when I put up the splash page?

Today I wanted to start writing the detailed use cases after having sketched most of the wireframe. I’ve been doing my usual research and here are the interesting documents I found for ‘use case’:

• Wikipedia Use Cases
• Writing Effective Use Cases (Alistair Cockburn)

Now, some developers might disagree with me, but I opted not to do all the different UML diagrams for 2 reasons:

1. As for any first version of a product, no matter how much I believe in its success, I prefer investing the least time possible when starting and instead focus on getting it built. When the application prooves to be successful, a complete rebuild can be made, inflicting a higher cost than just upgrading it, but that’s negligeable since it would pay for itself.
2. I don’t have a complete knowledge of UML to say that it would be a perfectly solid one. I’d rather concentrate on learning more about it instead of playing with it.

That been said, Writing Effective Use Cases inspired me to make an XML for writing those use cases so they could be parsed later on to write documentation, usability tests, etc. And those of course will be reusable.

Here is a sample of the first 3 use cases I wrote.

<usercases>
	<usecase>
		<name>Use Case 1</name>
		<description>Register User</description>
		<sitting>Instant</sitting>
		<primaryactor>User</primaryactor>
		<scope>System, Beanstream API</scope>
		<level>User goal</level>
		<stakeholders>
			<stakeholder>User</stakeholder>
			<interest>Getting a membership access</interest>
			<stakeholder>System</stakeholder>
			<interest>Saving all valid user data and creating account</interest>
			<stakeholder>Company</stakeholder>
			<interest>Getting paid</interest>
		</stakeholders>
		<minimalguarantee>Sufficient validation to detect wrong inputs and failed payments</minimalguarantee>
		<successguarantee>Beanstream API has acknowledged the purchase, the user's account is created</successguarantee>
		<scenario>
			<step>User selects membership plan</step>
			<step>System gets billing information from user</step>
			<step>System sends data to Beanstream API</step>
			<step>Beanstream API bills user</step>
			<step>System creates user account</step>
			<step>System writes cookie and session for user</step>
			<step>System forwards user to account</step>
		</scenario>
		<extensions>
			<extension>
				<extends>2</extends>
				<xcase>User enters invalid billing information</xcase>
				<step>System alerts user of the error(s)</step>
				<step>System gets new billing information from user</step>
			</extension>
			<extension>
				<extends>4</extends>
				<xcase>Beanstream API transaction fails</xcase>
				<step>System alerts user of the error(s)</step>
				<step>System gets new billing information from user</step>
			</extension>
		</extensions>
	</usecase>

	<usecase>
		<name>Use Case 2</name>
		<description>Log User</description>
		<sitting>Instant</sitting>
		<primaryactor>User</primaryactor>
		<scope>System</scope>
		<level>User goal</level>
		<stakeholders>
			<stakeholder>User</stakeholder>
			<interest>Accessing his account</interest>
			<stakeholder>System</stakeholder>
			<interest>Identifying and authorizing the user</interest>
		</stakeholders>
		<preconditions>
			<precondition>User must have a valid account</precondition>
		</preconditions>
		<minimalguarantee>Sufficient validation for existing account and right credentials</minimalguarantee>
		<successguarantee>System has authorized access, user gets access to account</successguarantee>
		<scenario>
			<step>System gets credentials from user</step>
			<step>System writes cookie and session</step>
			<step>System logs access datetime</step>
			<step>System redirects user to account</step>
		</scenario>
		<extensions>
			<extension>
				<extends>0</extends>
				<xcase>System validates session</xcase>
				<step>System redirects to account</step>
			</extension>
			<extension>
				<extends>1</extends>
				<xcase>User enters invalid credentials</xcase>
				<step>System alerts user of the error(s)</step>
				<step>System gets new credentials from user</step>
			</extension>
		</extensions>
	</usecase>

	<usecase>
		<name>Use Case 3</name>
		<description>Reset Password</description>
		<sitting>Instant</sitting>
		<primaryactor>User</primaryactor>
		<scope>System</scope>
		<level>User goal</level>
		<stakeholders>
			<stakeholder>User</stakeholder>
			<interest>Retrieving access to his account</interest>
			<stakeholder>System</stakeholder>
			<interest>Making sure the request comes from the real account holder</interest>
		</stakeholders>
		<preconditions>
			<precondition>User must have a valid account</precondition>
		</preconditions>
		<minimalguarantee>Sufficient validation to check existing account</minimalguarantee>
		<successguarantee>System resets password</successguarantee>
		<scenario>
			<step>System gets email address from user</step>
			<step>System sends confirmation email to user</step>
			<step>User follows email's link</step>
			<step>System resets password</step>
			<step>System emails new password to user</step>
			<step>System logs action</step>
		</scenario>
		<extensions>
			<extension>
				<extends>1</extends>
				<xcase>User enters invalid email</xcase>
				<step>System alers user with error(s)</step>
				<step>System gets new email address from user</step>
			</extension>
			<extension>
				<extends>3</extends>
				<xcase>User follows invalid link</xcase>
				<step>System alers user with error(s)</step>
				<step>System gives option to manually enter the confirmation code</step>
			</extension>
			<extension>
				<extends>3</extends>
				<xcase>User enters invalid confirmation code</xcase>
				<step>System alers user with error(s)</step>
				<step>System gives option to manually re-enter the confirmation code</step>
			</extension>
		</extensions>
	</usecase>
</usecases>

So here is the project’s timeline. I will update this as things get done and hopefully stick to it.

Phase 1: (1 week)
deadline: June 7th

- Sketch and perfect the wireframe (5 days)
- Define the use cases (5 days)
- Install Trac/SVN, setup CakePHP (1 day)
- XHTML the wireframe (2 days)
- XML the use cases (2 days)
- Select a name (7 days)

Phase 2: (2 weeks)
deadline: June 21st

- Design web app (11 days)
- XHTML web app (3 days)
- Install, test and optimize the required vendor classes (2 days)
- Code the required extra classes (5 days)
- Draw the general architecture (site, web app, web services, database, etc.) (1 day)
- Write the teaser page to recruit early-users (1 day)
- Design the database (2 days)

Phase 3: (4 weeks)
deadline: July 19th

- Design and implement teaser page (2 days)
- Develop web app (28 days)
- Design site (6 days)
- Design logo (10 days)
- Copy writing for the website (12 days)
- XHTML website (5 days)

Phase 4: (2 weeks)
deadline: August 2nd

- Testing (14 days)
- Add new third parties (10 days)
- Write and distribute press release (2 days)
- Launch