Announcing release of ChameleonForms 2.0.0 and new documentation site

I’m somewhat more subdued¬†with my excitement for announcing this than I was for 1.0. In fact I just had a chuckle to myself in re-reading that post ūüôā (oh and if you were wondering – did Matt and I enjoy Borderlands 2? Yes we very much did, it’s a great game).

Nonetheless, there is some really cool stuff in ChameleonForms 2.0 and I’m particularly excited about the new PartialFor functionality, which I will describe below. My peak excitement about PartialFor was months ago when the code was actually written, but Matt and I have had a particularly busy¬†second half of the year with our work roles expanding in scope and a healthy prioritisation of our personal lives so it took a while to get our act together and get the code merged and released.

There have been a range of point releases that added a bunch of functionality to ChameleonForms since the 1.0 release and before this 2.0 release. You can peruse the releases list to see the features.

New docs site

I’ve taken¬†the lead (as well as a bunch of advice – thanks mate) from Jake Ginnivan¬†and moved the documentation for¬†ChameleonForms to Read the Docs. The new documentation site is now generated from files in the source repository’s docs folder. This is awesome because it means that the documentation can be tied to current state of the¬†software – no more documentation that is ahead or behind and pull requests can now contain documentation changes corresponding to the code changes.

For those who are curious the process I followed to migrate from GitHub wiki to Read the Docs was:

  1. Clone the wiki
  2. Move all the files into the docs folder of the repository
  3. Add a mkdocs.yml file to the root of the repository with all of the files¬†(this means I need to keep a list of the files in there, but I don’t mind since it gives me control of the menu, you can omit the mkdocs.yml file if you want and it¬†alphabetically places all of the files in the menu)
  4. Sign up for Read the Docs and create a new project linked to the GitHub repository
  5. Enable the fenced code markdown extension
  6. Change all internal documentation links to reference the .md file (in my case I had to search for all links to wiki/* and remove the wiki/ and add in the .md)
  7. Change any occurrences of ```c# with ```csharp (GitHub supports using c# for the fenced code snippet, but mkdocs doesn’t)
  8. Check all of the pages since some of them might render weirdly – I had to add some extra spaces between paragraphs and code blocks / bullet lists for instance since the markdown parser is slightly different

There are a bunch of different formats that give more flexibility that Read the Docs supports (e.g. restructured text), but I’m very happy with the markdown support.

2.0 minor features and bug fixes

Check out the release notes for the 2.0 release to see a bunch of minor new features and bug fixes that have been contributed by a bunch of different people Рthanks to everyone that contributed! It always give Matt and I a rush when we receive a pull request from someone :).

PartialFor feature

This is the big feature. A few breaking changes went into the 2.0 release in order to make this possible. This is the first of the extensibility features we have added to ChameleonForms.

Essentially, it allows us to contain a part of a form in a partial view, with full type-safety and intellisense. The form can be included directly against a form or inside a form section. This makes things like common parts of forms for create vs edit screens possible. This allows you to remove even more repetition in your forms, while keeping a clean separation between forms that are actually separate.

The best way to see the power of the feature in it’s glory is by glancing over the acceptance test for it.¬†The output should be fairly self explanatory.

There is also a documentation page on the feature,

Is ChameleonForms still relevant?

We were very lucky to be included in Scott Hanselman’s NuGet package of the week¬†earlier this year. The comments of Scott’s post are very interesting because it seems our library is somewhat¬†controversial. A lot of people are saying that single page applications and the increasing prevalence of JavaScript make creating forms in ASP.NET MVC redundant.

Matt and I have spent a lot more time in JavaScript land than MVC of late and we concede that there is certainly a lot more scenarios now that don’t make sense to break out MVC. That means ChameleonForms isn’t as relevant as when we first started developing it.

In saying that, we still firmly believe that there are a range of scenarios that MVC is very much appropriate for. Where you don’t need the flexibility of an API and/or you need pure speed of development (in particular developing prototypes) and/or you’re building CRUD applications or heavily forms-based applications (especially where you need consistency of your forms) we believe MVC + ChameleonForms is very much a good¬†choice and often is the best¬†choice.

Testing AngularJS directives using Approval Tests

I recently had an application I was developing using AngularJS that contained a fair number of directives that were somewhat complex in that the logic that backed them was contained in services that called HTTP APIs. The intent was to provide a single JavaScript file that designers at the company I was working at could include and then build product pages using just HTML (via the directives). I needed to provide some confidence when making changes to the directives and pin down the behaviour.

As explained below, I ended up doing this via approval tests and I’ve published how I did it on GitHub.

Why I wanted to use Approval Tests

In order to test these directives I didn’t want to have to perform tedious DOM inspection code to determine if the directives did what I wanted. Most AngularJS directive testing examples you will find on the Internet tell you to do this though, including the official documentation.

Side note: in my research I stumbled across the ng-directive-testing library, which I feel is an improvement over most example code out there and if you do want to inspect the DOM as part of your testing I recommend you check it out.

This style of testing works fine for small, simple directives, but I felt would be tedious to write and fragile for my use case. Instead, I had an idea that I wanted to apply the approval tests technique.

I use this technique when I have a blob of JSON, XML, HTML, text etc. that I want to verify is what I expect and pin it down without having to write tedious assertions against every aspect of it Рhence this technique fitted in perfectly with what I wanted to achieve with testing the directives.

How I did it

Given that directives need the DOM it was necessary to run the tests in a web browser. In this case I decided to do it via Karma since I was already using Node JS to uglify the JavaScript.

ApprovalTests requires access to the filesystem in order to write the approval files and then access to open processes on the computer to pop open a diff viewer if there is a difference in the output. This is not possible from the web browser. Thus, even though there is a JavaScript port of ApprovalTests (for NodeJS) I wasn’t able to use it directly in my tests.

While contemplating my options, it occured to me I could spin up a NodeJS server to run the approvals code and simply call it from the browser – it’s not much different to how Karma gets test results. After that realisation I stumbled across approvals-server – someone had already implemented it! Brilliant!

From there it was simply a matter of stitching up the code to all work together – in my case using Grunt as the Task Runner.

Example code

To that end, I have published a repository with a contrived example that demonstrates how to test a directive using Approval Tests.

The main bits to look at are:

  • gruntfile.js – contains the grunt configuration I used including my Grunt tasks for the approval server, which probably should be split into a separate file or published to npm (feel free to send me a PR)
  • app/spec/displayproducts.directive.spec.js – contains the example test in all it’s glory
  • app/test-helpers/approvals/myapp-display-products-should-output-product-information.approved.txt – the approval file for the example test
  • app/test-helpers/approvals.js – the code to get name of currently executing Jasmine 2 test and the code to send an approval to the approval server
  • app/test-helpers/heredoc.js – a heredoc function to allow for easy specification of multi-line markup
  • app/test-helpers/directives.js – the test code that compiles the directive, cleans it up for a nice diff and passes it to be verified

Notable bits

Style guide

If you are curious about why I wrote my Angular code the way I have it’s because I’m following John Papa’s AngularJS style guide, which I think is very good and greatly improves maintainability of the resulting code.

Taming karma

I managed to get the following working for Karma:

  • Watch build that runs tests whenever a file changes – see the karma:watch and dev tasks
  • Default¬†build including tests – see the karma:myApp and default tasks
  • A build that pops up a Chrome window to allow for debugging – see the karma:debug and debugtests takss

Simultaneous approval server runs

I managed to allow for the dev task to be running while running default by including the isPortTaken code to determine if the approvals server port is already taken.

Side note: if you are using this code across multiple projects consecutively then be careful because the approval server might be running from the other project. A way to avoid this would be to change the port per project (in both gruntfile.js and approvals.js.

Improved approval performance on Windows

I found that the performance of the approvals library was very slow on Windows, but with some assistance from the maintainers I worked out what the cause was and submitted a pull request. The version in npm has been updated, but there are currently no updates to approvals-server to use it. To overcome this I have used the npm-shrinkwrap.json file to override the version of the approvals library.

Get currently running test name in Jasmine 2

I wanted the approval test output file to be automatically derived from the currently-running test name (similar to what happens on .NET). It turns out that is a lot harder to arhieve in Jasmine 2, but with some Googling/StackOverflowing I managed to get it working as per the code in the approvals.js file.

Cleaning up the output markup for a good diff

AngularJS leaves a bunch of stuff in the resulting markup such as HTML comments, superfluous attributes and class names, etc. In order to remove all of this so the approved file is clean and in order to ensure the whitespace in the output is both easy to read and the same no matter what browser is being used I apply some modifications to the markup as seen in directives.js.

Easily specifying multi-line test markup

I pulled in a heredoc function I found on StackOverflow as seen in heredoc.js and used in the example test, e.g.:

DirectiveFixture.verify(heredoc(function () {/*    
    <myapp-display-products category="car" product="car">
        <div>{{car.name}}</div>
    </myapp-display-products>
*/}));

This is much nicer than having to concatenate one stirng per line or append a \ character at the end of each line, both of which aren’t handled nicely by the IDE I’m using.

 

Deployment and Development using PhoneGap Build for a Cordova/PhoneGap app

I recently worked on a project to¬†build a mobile app in iOS and Android. The team decided to use¬†Cordova¬†and¬†Ionic Framework¬†to build it. I highly recommend Ionic – it’s an amazingly productive and feature-rich framework with a really responsive development team! It enabled us to use AngularJS (and the application structure and maintenance advantages that affords) and really sped up the development time in a number of ways. We were also able to get an almost-native performance experience for the application using features like¬†collection repeat¬†(virtualised list) as well as the in-built performance tuning the Ionic team have performed.

When it was time to set up an automated build pipeline we settled on using PhoneGap Build and its API which we then called from OctopusDeploy as part of a NuGet package we created.

PhoneGap Build¬†is Adobe’s cloud build service that can build iOS, Android, Windows Phone etc. apps for you based on a zip file with your¬†Cordova/PhoneGap¬†files.

We decided on PhoneGap Build because:

  • It has been around a while (i.e. it’s relatively mature)
  • It’s backed by a big name (Adobe)
  • It has a decent API
  • It meant we didn’t have to set up a Mac server, which would have been a pain
  • It’s free for a single project ūüôā

This post covers how we set up our automated deployment pipeline for our native apps using OctopusDeploy and PhoneGap Build. I have also published a¬†companion post that covers everything you need to know about PhoneGap Build¬†(read: what I wish I knew before I started using PhoneGap Build because it’s darn confusing!).

Deployment pipeline

This is the deployment pipeline we ended up with:

  1. Developer commits a change, submits a pull request and that pull request gets merged
  2. Our TeamCity continuous integration server compiles the server-side code, runs server-side code tests and runs client-side (app) tests
  3. If all is well then a NuGet package is created for the server-side code and a NuGet package is created for the app – both are pushed to OctopusDeploy and the server-side code is deployed to a staging environment (including a web server version of the app)
    • Native functionality wouldn’t work, but you could access and use the bulk of the app to test it out
    • To test native functionality developers could easily manually provision a staging version of the app if they plugged a phone into their computer via a USB cable and executed a particular script
  4. The product owner logs into OctopusDeploy and promotes the app to production from staging whereupon OctopusDeploy would:
    • Deploy the server-side code to the production environment
    • Use the PhoneGap Build API to build an iOS version with an¬†over-the-air compatible provisioning profile
    • Download the over-the-air¬†.ipa
      • Upload this¬†.ipa¬†to Azure blob storage
      • Beta testers could then download the latest version by visiting a html page linking to the package (see below for details)
    • Build the production Android and iOS versions of the app using the PhoneGap Build API
  5. The product owner then logs on to the PhoneGap Build site, downloads the .ipa / .apk files and uploads them to the respective app stores

End result

I personally wasn’t completely satisfied with this because I didn’t like the manual step at the end, however, apparently app store¬†deployments are¬†stuck in the 90s so unless you want to screen-scrape the respective app stores you have to manually log in and upload the deployment file. Regardless, the end result was good:

  • Production deployments were the same binaries/content that were tested on the CI server and that had been successfully deployed and manually tested in the staging environment
  • We could go from developer commit to production in a few minutes (ignoring app store approval/roll-out times; remember – stuck in the 90s)
  • The product owner had complete control to deploy any version of the app he wanted to and it was easy for him to do so

How we did it

I have published the NodeJS script we used to automate the PhoneGap Build API to a Gist. It contains a very reusable wrapper around a PhoneGap Build API NodeJS library that makes it dead easy to write your own workflow in a very succinct manner.

To generate the zip file the NodeJS script used we simply grabbed the www directory (including config.xml) from our repository and zipped it up. That zip file and the NodeJS scripts went into the NuGet package that OctopusDeploy deployed when the app was promoted to production (we made this happen on a Tentacle residing on the OctopusDeploy server). I have published how we generated that NuGet package to a Gist as well (including how to set up over-the-air).

Further notes

Some notes about our approach:

  • Because we were able to make our backend apis deploy to production at the same time as generating our production app packages we were assured that the correct version of the backend was always available when the apps were deployed to the respective app stores.
  • It was important for us to make sure that our backend api didn’t have any breaking changes otherwise existing app deployments (and any lingering ones where people haven’t updated) would break – usual API management/versioning practices apply.
  • Whenever we wanted to add new Beta testers the provisioning profile for over-the-air needed to be re-uploaded to PhoneGap Build
    • Unfortunately, PhoneGap Build requires you upload this alongside the private key – thus requiring someone with access to PhoneGap Build, the private key and the iOS developer account – it’s actually quite a convoluted process so I’ve also published the instructions we provided to the person responsible for managing the Beta testing group to a¬†Gist¬†to illustrate
  • We stored the passwords for the Android Keystore and iOS private keys as¬†sensitive variables in OctopusDeploy¬†so nobody needed to ever see them after they had been generated and stored in a password safe
    • This meant the only secret that was ever exposed on a regular basis was the password-protected over-the-air private key whenever a Beta tester was added

TestFlight

While we didn’t use it, it’s worth pointing out¬†TestFlight. It’s a free service that gives you the¬†ability¬†to more easily keep track of a list of testers, their UUIDs, informing them of updates and automatically creating and hosting the necessary files for over-the-air. It contains an API that allows you to upload your¬†.ipa¬†to and it should be simple enough to take the scripts I published above and upload a¬†.ipa¬†to TestFlight rather than Azure Blob Storage if you are interested in doing that.

Final Thoughts

PhoneGap Build is a useful and powerful service. As outlined in my companion post, it’s very different to Cordova/PhoneGap locally and this, combined with confusing documentation, resulted in me growing a strong dislike of PhoneGap Build while using it.

In writing my companion post however, I’ve been forced to understand it a lot more and I now think that as long as you have weighed up the pros and cons and still feel it’s the right fit then it’s a useful service and shouldn’t be discounted outright.

ChameleonForms 1.0 Released

I’m incredibly excited and proud to finally announce the release of 1.0 of the ChameleonForms library I’ve been working on with Matt Davies. My blog has been fairly quiet the last couple of months while I’ve poured time and energy in finally getting ChameleonForms to 1.0.

WOOOOOOOOOOOOOOOO

(Apologies; I’m releasing months of built-up anticipation and excitement here).

I’m biased of course, but I think this library is amazing to use and results in immensely more maintainable form generation code when using ASP.NET MVC. It extends on the knowledge that MVC developers would have in generating forms using the (already pretty awesome) built-in stuff, but adds the things I think are missing. For me, this library epitomises over 7 years of exploration in the best way to do web-based forms and I’m excited to be able to share the beginnings of my current vision via this library.

What is ChameleonForms?

In short, ChameleonForms takes away the pain and repetition of building forms with ASP.NET MVC by following a philosophy of:

  • Model-driven¬†defaults (e.g. enum is drop-down,¬†[DataType(DataType.Password)]¬†is password textbox)
  • DRY¬†up your forms – your forms will be quicker to write and easier to maintain and you won’t get stuck writing the same form boilerplate markup form after form after form
  • Consistent¬†– consistency of the API and form structure within your forms and consistency across all forms in your site via templating
  • Declarative¬†syntax – specify how the form is structured rather than worrying about the boilerplate HTML markup of the form; this has the same beneficial effect as separating HTML markup and CSS
  • Beautiful, terse, fluent APIs¬†– it’s a pleasure to read and write the code
  • Extensible and flexible¬†core – you can extend or completely change anything you want at any layer of ChameleonForms and you can drop out to plain HTML at any point in your form for those moments where pre-prepared field types and templates just don’t cut it

More info.

What are the big improvements in 1.0?

We’ve been releasing pretty often so that depends on what version you are currently using, but these are the most important things:

  • Extensive usage across a number of production websites – we are happy that this library is mature, stable and ready for prime-time
  • Twitter Bootstrap 3 Template out-of-the-box supported by a NuGet package to get you up and running faster – this is HUGE for a number of reasons:
    • Bootstrap is pretty darn popular right now so this is immediately useful to a lot of people
    • In creating this template we had to do some pretty sophisticated changes to allow the template to drive a lot of changes unobtrusively to the form structure you are adding in your views – this is great because it means it’s really easy for you to create your own form templates and accomplish similarly complex transformations of your form markup
    • The ASP.NET MVC templates that come with Visual Studio 2013 come with Bootstrap by default now – and boy do they have gross repetitive boilerplate in them, which you can clean right up using this library
    • The vision that we have for this library is coming to fruition, which is personally gratifying – this is a beautiful demonstration of being able to declaratively specify the structure of your form and then completely change the markup/template of your form across a whole application with a single line of code when it changes
  • Really comprehensive documentation of everything in the library – we’ve spent many hours writing up the documentation – the idea was to make it comprehensive, but accessible/terse; hopefully we’ve met that goal!
  • Really solid code coverage to help prevent regressions or breaking changes as well as some refactorings that give us a solid codebase to continue with the other features we want to add – hopefully this can support us into the future with minimal breaking changes

How can I get it?

Checkout the GitHub release or go to NuGet.

Versioning

From this point on we are following semver thanks to the GitHubFlowVersion project. The fourth number in the NuGet version number is actually build metadata.

Borderlands 2 here I come!

Over a year ago now (yes it’s been a long journey – our first NuGet package was published on November 1, 2012) Matt Davies and I made an agreement to each other that neither of us would play the recently released Borderlands 2 game (we were both huge fans of Borderlands so this was a big deal) until we released 1.0 of ChameleonForms so that we would remain focussed on it and not get distracted. Now, while we both didn’t realise that it would take this long and while the last couple of months have seemed like forever (I’m pretty sure we had a phone conversation at least once a week where one of us would say “dude, we are sooooo close to 1.0 and BORDERLANDS 2!”) we are both incredibly proud of the library and are happy with what we’ve managed to get into 1.0.

Needless to say, we will probably be taking a break from open source for a few weekends to play Borderlands 2 ūüôā

We hope you enjoy using the library!

As usual hit us up with issues and pull requests on GitHub – they make our day ūüôā

NQUnit update

I’ve just pushed a new version of NQUnit and NQUnit.NUnit (1.0.5). This update updates the libraries to use the latest version of Watin, NUnit, jQuery and QUnit.¬†I’ve also made some documentation improvements. You can see the latest code and documentation on GitHub.

It’s worth noting that while NQUnit still works perfectly well, there is a better option that I usually recommend in preference: Chutzpah. I’ve updated NQUnit as per pull requests that the library received – while there are still people using the library I’m happy to keep it up to date.

WebApi.Hal

Lately I’ve been having a bit of fun learning ASP.NET Web API and I’ve been working with Jake Ginnivan to come up with a library that allows you to follow the HATEOS principle of REST using it. We’ve been doing this by implementing a specification that allows you to specify hypermedia for JSON and XML called HAL.

Jake did some awesome work to start it off and more recently I came along and tried to make the specification of the hypermedia as unobtrusive as possible.

It’s very rough around the edges, and definitely a work-in-progress, but if you are interested feel free to check out the latest work that I’ve been doing on it at:¬†https://github.com/robdmoore/Hal.PlayAround.

To see it in action simply fire up the site, the homepage is a HTML page that allows you to interact with the dummy API I created via a set of HTML forms (the actual requests are made with JavaScript). If you inspect the PersonController class you will see that I barely had to add anything to it to get the hypermedia in there.

Enjoy!

If you want to be a good web developer

I’ve been doing web development professionally since 2004 and in that time I’ve learnt a lot and I consider myself to be a good web developer. I’m mostly self-taught, learning most of what I know from research on the Internet, in combination with experimentation, to both find out the way things work and figure out the best ways to use them.

While the Internet is an amazing resource, there is a lot of bad information out there and you should always be careful when taking information at face value. I’ve compiled a list of links to information that I personally think is important if you want to be a good web developer (or designer).

It’s really difficult trying to bring to mind all of the most important things that I know so there is probably quite a bit missing, but this is a good start. I may well follow this post with another one later with things I’ve missed. If you think there is something really important that I’ve missed fell free to leave a comment.

Without further ado (written in C# for fun, if you don’t know C# it’s ok just click on the links):

Continue reading “If you want to be a good web developer”