Tonight I made the trip back up to Coventry for a NxtGenUG event Dave Verwer from Shiny Development gave a talk on Ruby on Rails.  Having used Rails in my final year project I had a 'good' understanding already, mainly on the security side, however Dave gave a great insight into Rails and made certain items which where unclear before (but due to time limited I didn't dive deep into Rails) became a lot clearer and I understand more how everything links together and how some things are possible.  Ruby is on my list of things to learn, but after tonight has moved up one or two places in important.

I also gave a nugget on the Imagine Cup - Slides are available here.  The nugget was based on my experience within this years Imagine Cup competition. I think the talk went well, maybe I went over everything a little bit too quick but it was good experience so hopefully next time I will be a little bit more relaxed - I had more time than I thought.

As for the event itself, it was well organised, very friendly. If you haven't attended a user group meeting before, I really recommend attending, they are very relaxing but informative at the same time, plus if you are unsure about anything then just drop the organisers a email and they should be happy to help you.  No matter where you are in the UK, there should be a user group within the local area, Microsoft have a list on their site.  If you have problems finding a group, let me know and I'll ask around for you.

Next event for me will be DDD5, then back to NxtGen for WPF.

Labels: NxtGenUG

Another one of the new features within SQL Server 2008 is the Change Data Capture (CDC).  CDC is designed to capture insert, update and delete activity on a SQL table and place the information into a separate relational table.  It uses an asynchronous capture mechanism that reads the transaction logs and populates the CDC table with the row's data which change.  The CDC table mirrors the column structure of the tracked table, together with metadata regarding the change.  In later releases, there is also going to be a audit feature allowing you to audit who accesses what information and when, based more on select statements.

One of the posts on the forums details how to setup CDC, it's also in books online. To enable CDC, the following two statements are required. Note, SQLServerAgent must be running for this to work.

use testDatabase
--Activate CDC
EXEC sys.sp_cdc_enable_db_change_data_capture
--IsDatabaseEnabled?
SELECT is_cdc_enabled FROM sys.databases WHERE name = 'testDatabase'
--Enable CDC on table
EXEC sys.sp_cdc_enable_table_change_data_capture @source_schema = 'dbo', @source_name = 'Table_1', @role_name = 'cdc_test'
--IsTableEnabled?
SELECT is_tracked_by_cdc FROM sys.tables WHERE name = 'table_1'

The sp_cdc_enable_table... statement enables the tracking of the changes for a table, parameters are required for setting the source schema, the source name which is the table you want to track, and the role name, if the role does not exist it will create it for you.

After executing these statements you will have a number of new tables within your database.  The tables are:

• cdc.captured_columns - Information about the captured columns being tracked.
• cdc.change_tables - Tables being tracked.
• cdc.ddl_history - Changes to the tracked table, details the command issued on the table.
• cdc.index_columns - Guessing this is tracked indexed columns.
• cdc.lsn_time_mapping - When a transaction starts and ends. Relates to rows in the tracked tables.

There is also a table for each table which is tracked, in my case I have just the one - cdc.dbo_Table_1_CT.  If we execute a change to the table, such as inserting a new record.

INSERT INTO Table_1 VALUES (1, 'First Test')

A new row is added to the table, with a record also being inserted into lsn_time_mapping.

__$start_lsn           __$end_lsn             __$seqval              __$operation __$update_mask id          TestMessage
0x0000001E0000006B001A NULL                   0x0000001E0000006B0018 2            0x03       1           First Test

If we update the table again using

UPDATE Table_1 SET TestMessage = 'Updated First Test' WHERE id = 1

Two records are inserted into the tracked table. One with the TestMessage 'First Test' (the original message) and the other with 'Updated First Test' both relating to a single transaction in the lsn_time_mapping table.  If we do

DELETE FROM Table_1 WHERE id = 1

We have a single row inserted into dbo_Table_1_CT saying which row was deleted.

Within the dbo_Table_1_CT there is a column __$operation.  This gives you the ID relating to the operation executed on the row.  2 = Insert. 3 = Before Update. 4 = Post Update. 1 = Delete.

If we change the design of the table while it is being tracked, a row is inserted into cdc.ddl_history with the command executed on the table.

source_object_id object_id   required_column_update ddl_command                    
---------------- ----------- ---------------------- --------------------------------
1077578877       1093578934  0                      ALTER TABLE dbo.table_1 ADD Status nvarchar(50) NULL

However, if we execute another insert command on the table, the new column does not appear in the result set/dbo_table_1_ct.  It would be nice to keep the two insync and not have to worry about updating the table if it is being tracked. To get the tables back insync, I had to disable it (capture_instance parameter relates to the columns being captured I think) and re-enable the tracking. However, in doing so I lost all of the previous information - Boo.  The commands to do this are:

EXEC sys.sp_cdc_disable_table_change_data_capture @source_schema = 'dbo', @source_name = 'Table_1', @capture_instance = 'all'
EXEC sys.sp_cdc_enable_table_change_data_capture @source_schema = 'dbo', @source_name = 'Table_1', @role_name = 'cdc_test'

In case you are worried about this using up too much space, there is a sys.sp_cdc_cleanup_change_table stored procedure which can be used to remove entries up to a point in time.  You could then link this to a Job to archive the information, or simply report and remove.

I think that pretty much covers it. Another really useful feature, I have saw one or two attempts at auditing database activities like this and they are a bit hit and miss, so this will solve a lot of heartache. Looking forward to seeing what happens with the future auditing features.

One concern:  If a table is tagged as encrypted, is the cdc table also encrypted?  Will have to research this at a later date.

SQL BOL: ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10de_1devconc/html/7d514b05-7c4b-4ef2-8f16-b720df3bf44a.htm

Labels: Katmai, SQL Server

The final for 2007 hasn't even been held yet and the theme for 2008 has been announced

“Imagine a world where technology enables a sustainable environment.”

Best get your thinking caps on.

One of the new features within SQL Server 2008 is Declarative Management Framework, a new policy based management framework allowing you to define what can or cannot be done on the server itself which can then be redistributed across your network of servers for more consistent management.  Content based on the June CTP.

Three main components make up the framework

• Policy management.  Create the policies from within SQL Management Studio.
• Explicit administration.  Select and manage targets for the policies.
• Automated administration. How and when the policies are evaluated.

Some of the important terminology includes:

• Facet.  Set of logical properties that model behaviour or characteristics.  There are a number of facets built into the server, each of these have a set of properties for which you can set a condition on based on what the policy is.  For example, there is a Surface Area Facet which allows you to set if a setting should be on or off. 
• Condition.  Expression of allowed states.
• Policy.  Condition and the expected behaviour.
• Policy group.  Group to help manage policies.  A policy can only belong to one group.
• Execution Mode.  How the policy will be executed. 

What is a Facet again?

Facet is one of the core concepts within the DMF.  They set out all of the properties which can be used as part of the condition/policy without you having to worry about how to locate and access the information.  When you create a new condition, you say which facet contains the information you want, and then set a condition to the properties you are interested in.

When it comes to creating a policy, the facet plays a part in what the policy can be targeted at.  The Multipart Name Facet can be targeted/applied to a number of different objects in the system while the surface area facet only applies at server level.  This makes managing the policy a lot similar as you only see the options which you are interested in.

If your unsure on where the property you are interested in is contained within which facet, you can right click on the object > policies > facet.  This will display all the related facets with all the properties set to match the selected object.  From here, you can also create a policy to define how it should be configured, makes it quick to setup a known baseline and tweak the policy from there.

What and How are they executed

DMF executes the created policies either on an ad-hoc basis, or more automated via Check on Schedule, Check On Changes or Enforce which are set when the policy is created, or changed at a later point.  Policies can also be set to disabled/enabled.  Policies can also be checked in test mode to make sure they are working correctly.  Check On Schedule uses a SQL Server Agent job to evaluate a policy.  Check On Changes uses event notification to evaluate based on when changes occur.  Enforce uses DDL triggers to prevent policy violations.  Notice, the first two do not prevent, just notify, while Enforce prevents the changes taking place.

Dan has done a really good write up on the details of enforce at Declarative Management- Behind Enforce. To summarise the main points.  Enforce works off a public DDL trigger which listens for all server and database events.  This is a single trigger and is updated with the events required which is based on the active polices, which in turn is based on the facet.  The DM engine runs inside the SQLCLR,  however it runs even if SQLCLR is set to Off as the Off mode still allows assemblies signed by Microsoft to execute.  An interesting point is that the DM is dependent on the DDL eventing and so can only enforce state on objects that are transactions.   If it is not a transaction, then it is not allowed to be enforce (option will be disabled) as it cannot be cleanly rollbacked if it fails.

How do you create a policy?

When creating a policy, the first thing you need to do is create a condition.  This is all built into SQL Management Studio, within the Object Explorer, under Management there is a new tree item for Policy Management. This contains all of the information relating to the facets, conditions and policies related to the connected server.

To create a new condition, right click Conditions or a Facet to display the create dialog, you will need to give it a name, choose the related Facet and then set the conditions.  You can do some really powerful expressions, however I do not think there is a way to do cross two facets.  Maybe giving this as a SQL statement would allow more flexibility.

Once the condition has been created, you can then attach a policy to that condition. Right click Policies and select new policy. You will need to give it a name, select the condition you just created, then select what the policy applies to, finally selecting the execution mode.  That's it, the policy is now setup to manage the server.  A policy can only have one related condition, however you can have a group of policies.

 As already mentioned, you can create a condition and related policy from an existing database object.

Where are they stored?

Policies are stored in the msdb database but also can be exported as a not very interesting XML file.  They can be accessed via views, or via SQL Management Studio.

Bug

There is currently a known bug in the CTP which means the DMF does not work. As mentioned, when the CLR is off only Microsoft signed assemblies can be executed, however the Declarative Management DLL did not get signed as part of the build process. When a policy is executed, it causes the following error:

Msg 10314, Level 16, State 11, Procedure sp_syspolicy_execute_policy, Line 25
An error occurred in the Microsoft .NET Framework while trying to load assembly id 65536. The server may be running out of resources, or the assembly may not be trusted with PERMISSION_SET = EXTERNAL_ACCESS or UNSAFE. Run the query again, or check documentation to see how to solve the assembly trust issues. For more information about this error:
System.IO.FileLoadException: Could not load file or assembly 'Microsoft.SqlServer.DmfSqlClrWrapper, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. Strong name validation failed. (Exception from HRESULT: 0x8013141A) ---> System.Security.SecurityException: Strong name validation failed. (Exception from HRESULT: 0x8013141A)
System.Security.SecurityException:

You need to enter the following command and restart services to allow SQL to execute non-signed/strong named assemblies.  The '89845DCD8080CC91' string is the Public Key Token for the assembly in case you was wondering.

"%ProgramFiles%\Microsoft Visual Studio 8\SDK\v2.0\Bin\sn.exe" -Vr Microsoft.SqlServer.DmfSqlClrWrapper,89845DCD8080CC91

Summary

The Declarative Management Framework is an excellent addition to the SQL Server toolset and I can see a lot of benefit for DBAs, no more going around 100 servers to check that the CLR is enabled, instead they can just execute an ad-hoc script or enforce it by default.

Dan Jones, who works on the SQL team, has lots of interesting information about this on his blog.

http://blogs.msdn.com/dtjones/default.aspx

SQL Books online:

ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10de_4deptrbl/html/ef2a7b3b-614b-405d-a04a-2464a019df40.htm

Labels: Katmai, SQL Server

Zi has just uploaded a Wiki for the DDD5 Geek Dinner.  Signup at http://www.zimakki.com/wiki/DDD5GeekDinner.ashx.  Should be a good night.

There is also a travel arrangements wiki http://www.zimakki.com/wiki/DDD5Travel.ashx if you have space in your car to take a few people.

I have to admit, Hack Day wasn't for me. The early afternoon was great, however when the hacking started the interest started to drop off for me for some reason.  So, I left :(  It was a shame, I had a few good ideas and things started to come together however I just wasn't feeling the staying overnight and hacking.  At home I stay up easily, however I just wasn't that interested yesterday. Oh well, you win some you lose some.  Everyone else seemed to be having a good time. 

Just arrived here at Alexandra Palace for HackDay. So far its very very different.  Sitting in the main hall which is darkened out with loads of tables in the middle of the hall with a large stage with the guys playing Halo.  Good news is, every table looks to have a series of power adapters underneath them so no worry about battery life - my laptop is currently hovering around the two hours left mark.

The first session was on "Your Inbox To Go - BBAuth and the Yahoo! Mail API" which how to link into the web services Yahoo offer for their Mail services. Sadly the wireless network went down because someone created their own BTOpenworld network - never mind.  The second session was based on BBC Backstage which demonstrated some of the prototypes developed using their API's.  They offer a lot, hopefully I will be able to hook into them.  Third session was based on the new Geo services coming from Yahoo. 

However half way thought this the roof was hit by lightening, and the skylights opened up in the main hall.  It's a little bit wet.  Still lightening and now everyone has been moved into the main foyer for there own safety.  So new blog post title:

HackDay a Washout

I think its pretty funny, I know its potentially dangerous but its not something which happens at every conference and will go down in history :)  However, it looks like we are going to be in here for some time.  Shame I haven't got wifi and my battery is running low.

WiFi back up, battery charged.  Everything back to 'normal'.

Tomorrow and Sunday I will be attending HackDay at Alexandra Palace.  Not really sure what to except, but i'm excited about it. Haven't got a clue what i'm going to create, hopefully some of the pre-sessions will inspire me.  Sure it will end up being some mashup.  I am planning on posting how I'm getting on throughout the weekend so keep an eye on the blog. 

In an effort to get up to speed on SQL Server 2008, I decided to write a series of posts on some of the new features, as and when they are included in the CTP releases.  This post relates to the new SQL Merge statement.  Now, I say new but this was included within SQL Server 2005 Beta 2 however was pulled from RTM.  This kind of thing is one of the reasons why the team are only including near-complete features in the CTP.

The feature allows you to merge two tables together based on a set of criteria. The Merge works by stating a table where all the data should be merged into; the table, or query, where the data is coming from; and the criteria to decide if they match.  Based on this, the query can perform an action for each row such as insert, update or delete.  However, the action performed depends on the result of a row being matched from one table to the other.  There are three different matched clauses:

• WHEN MATCHED THEN
• Rows that meet the criteria.
• WHEN [TARGET] NOT MATCHED THEN
• Rows that do not match another row in the target table
• WHEN SOURCE NOT MATCHED THEN
• Rows that do not match another row in the source table

You can also set additional conditions on the matched clause, with different outcomes for each clause, for example
WHEN MATCHED AND tab.aValue > 100 THEN
WHEN MATCHED AND tab.aValue < 100 THEN

This example will demonstrate how the MERGE statement works.  Nothing interesting but helps the explanation.

Firstly, I have created two tables and inserted two rows into each table.

CREATE TABLE DataTable1 (ID int, DataValue varchar(50), Msg varchar(50))
CREATE TABLE DataTable2 (ID int, DataValue varchar(50), Msg varchar(50))
GO

INSERT INTO DataTable1 VALUES (1, 'TDV1', 'Inserted'), (2, 'TDV2', 'Inserted')
INSERT INTO DataTable2 VALUES (1, 'TDV3', 'Inserted'), (3, 'TDV4', 'Inserted')
GO

This is when the Merge query to merge DataTable1 into DataTable2.

MERGE DataTable2 AS mainData
USING (SELECT ID, DataValue FROM DataTable1) otherData
ON (mainData.ID = otherData.ID)
WHEN MATCHED THEN UPDATE SET Msg = 'MATCHED'
WHEN TARGET NOT MATCHED THEN INSERT VALUES (ID, DataValue, 'TARGET NOT MATCHED')
WHEN SOURCE NOT MATCHED THEN UPDATE SET Msg = 'SOURCE NOT MATCHED';
GO

Breaking this down:
MERGE defines the target table which is the target for any insert, updates or deletes which are defined in the matched clauses.
USING specifics the source table. 
ON specifics the match condition.
When a row is matched, we change the message in the row from Inserted to Matched.
If the row (in the source table) does not match another row in the target table, we insert the row into the target table and set the message to Target not matched
If the row (in the target table) does not match another row in the source table, we update the message to Source not matched.

The actual statement which can be executed with the matched block has some restrictions, for example I couldn't do WHEN TARGET NOT MATCHED THEN DELETE.

The result of this is merge is that DataTable2 has three rows:

1    TDV3    MATCHED
2    TDV2    TARGET NOT MATCHED
3    TDV4    SOURCE NOT MATCHED

Because we are matching on ID, ID=1 matched in both tables and because all changes affect the target table, the TDV3 row was kept and updated disregarding TDV1 row.  TDV2 was in our source table, however did not have a relating row in the target table and so was caught by our Target not matched clause. Finally, TDV4 was in our target table, but no row in the source data being merged in resulting in it failing under the source not matched clause.

In summary, this is a useful feature. Going to be a great help for people managing data warehouses.  It would be useful for synchronizing data, however I think there are more appropriate tools for that now (or to be) included. 

Reference in SQL BOL (Books Online) : ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10de_6tsql/html/c17996d6-56a6-482f-80d8-086a3423eecc.htm

Full SQL Query Statement : http://blog.benhall.me.uk/Code/Sql/SqlMerge.txt

The Apple Worldwide Developer Conference keynote was held today, this is a summary of the important announcements.

1. Safari 3 Public Beta. Safari is now a available on both OSX and Windows with a public beta released today.  Apple are promoting quicker loading of pages (2x faster than IE7).  The beta is also available for OSX, and will be included within Leopard and as the browser on the iPhone.  Download now @ http://www.apple.com/safari/
2. Electronics Arts (EA) are going to release games for the Mac.
3. Leopard features demo'ed
1. New menu bar, new dock, new feature called 'Stacks'.  Stacks helps you keep a clean desktop by stacking the files in the dock and will fan out, or pop up in a grid.  Now all of your downloads are viewed this way. Keeps everyone happy with the GTD approach.
2. New Finder, improved way to share documents
3. 64Bit top to bottom
4. Boot camp built in, no more burning drivers CD to install
5. 'Spaces'. Group applications in separate spaces, instantly switch between them.  Like desktop managers on Linux.
6. 300 new features in Leopard.
7. Basic costs $129.  Premium edition costs $129 ;)
4. iPhone will allow developers to write applications!  Based on Safari engine, uses Web 2.0 + Ajax, but integrate perfectly with iPhone services so they can make a call, check email, lookup your location all accessed via the web. No SDK required. Ajax on a mobile sounds impressive. Something which is missing on Windows Mobile.

Stacks sounds very useful and cool. Safari browser on Windows - do we really need another browser? However speed improves sound good and it will leave developers with no excuse for not having a Safari compatible site (I guess they are based on the same rendering engine) which is very good.  As for iPhone, great that they are allowing applications to link into the services, will be interesting to see their security model/approach, having Ajax support is definitely something which will attract developers. 

This week, Microsoft released SQL Server 2008 (Katmai) June CTP at TechEd. The ISO, or extractable zip, are available online at connect, just require registration to download, and are around 800mb in size.

I installed the CTP within a Virtual Machine running Windows 2003 Enterprise SP1.  On my first attempt, I was having a "Error 193" with regard to installing .Net 2.0 Framework.  It sounded like the dvd was having a hard time being read, maybe it was a bad DVDR, so I switched to using the mounted ISO and it installed fine, nothing overly special.  The installation process is very similar to SQL Server 2005 and installed happily under the VM machine.

Microsoft have said they are taking a different approach with their CTP releases for 2008. Instead of having lots of half complete features, they are focusing their efforts a lot more on having end-to-end features/scenarios completed (as much as they can be) and including them within the CTP and leaving out the half done features.  This means when you are trying a new feature, not only should it work, but also be a true presentation of what it will be like within the final release.  The team says its to help focus the feedback and development of the product, so not a repeat of some of the issues with 2005 Beta. The product overview white paper defines their vision for the release.

With the 2008 release, they have identified four key areas for features and improvements:

• Mission-Critical Platform
• Dynamic Development
• Beyond Relational Data
• Pervasive Business Insight

But what does that all mean?  I have highlighted some of the features from the white paper I think sound useful.

Mission-Critical platform revolves around secure and available data. 2008 includes transparent data encryption where entire databases, data files and log files can be encrypted within the server by simply 'tagging' the item.  Another new feature is the Declarative Management Framework which allows policies to be setup for managing the database instead of using scripts.  This sounds like it could solve many problems for companies, as it allows for a policy to span all the servers within the business, or have a policy for each 'type' of server, so development would be different to production allowing for a consistent management and server environment.  This should also make life easier for the replicating live servers within a test environment as it would just require the policy to be included/published.

Dynamic Development is based around allowing developers to work at the business entities layer instead of tables and columns, which is the ADO.NET Entity Framework, this isn't dependent on Katmai but is in the same marketing wave.  There is also improved support for synchronisation of data for those "Occasionally connected systems", this includes a new SQL Merge statement.

Beyond Relational Data, includes support for a seamless transition between relational and nonrelational data.  By including a new FileStream data type, you can now store BLOBS (Binary Large Object) to be stored in the file system allowing them to be managed by the database, but stored on the file system.  This together with integrated full text search allows for the database to store more kinds of data, for example documents or videos.  Along with this, Microsoft are including Location data support for creating location aware applications. This then can be integrated into Virtual Earth, which could produce some amazing business line applications very quickly and effectively and increase the uptake of business mashups.  This is not included within the June CTP.

Pervasive Business Insight is based around improving the data warehousing technology, together with new reporting functionality.  Two main new reporting features are the ability to author reports directly within Word or Excel and publish them to SharePoint.  The second being that Microsoft have acquired Dundas's data virtualisation technology and it will be included within 2008. Migration from Dundas to SQL Server 2008 will be 'seamless' and you can start working with their components today in preparation. Very cool.

Not sure how long Microsoft have been doing this, but there samples for SQL Server are now on CodePlex - http://www.codeplex.com/MSFTDBProdSamples

Sounds like a very interesting time, lots of different things to look at.  It's applications like this which are one of the reasons why I enjoy working in the real world, as being a student you really can't see/understand the benefit as much.

I just saw on the Mix RSS feed that the dates for the UK ReMix conference dates has been announced - September 11 - 12.  Guessing these are still unconfirmed, but does fit in with what Ian Moulster has said on his blog.

If you have not heard, ReMix is going to be a Mix like conference running at various locations around the world.

Source : http://visitmix.com/Blogs/Joshua/attend-a-remix-event-near-you/

Just a few announcements which I have saw on some posts, but I thought I would post some information here as well...

Acropolis CTP has been released - "set of components and tools that make it easier for developers to build and manage modular, business focused, client .NET applications." http://windowsclient.net/Acropolis/

SQL Server 2008 (Katmai) June CTP is now available for download from connect. https://connect.microsoft.com/SQLServer

More information on those two during the week when I have a chance to download and play... always playing never actually releasing anything :(

Bad news - Orcas has been officially named "Visual Studio 2008" - No release this year? http://www.danielmoth.com/Blog/2007/06/visual-studio-2008-stack.html

Dinis Cruz, UK Security Consultant/OWASP Chief Evangelist, has got a couple of UK based events coming up.  If you have saw Dinis present before, he has done DDD a couple of times and the Ed Gibson Roadshow, you will know what to expect, if you haven't saw him before he is very passionate about security and knows lots of very cool stuff and gets it across in a very important way.  Have to say he is the one who got me into security.

Source Code Security Training
Ounce Labs on 12^th and 13^th of June in London at the Thistle Marble Arch and covers topics such as Performing Source Code Reviews, Identifying Vulnerabilities in Code and Writing Exploits.  Sounds like a very cool event.

More Information at http://www.ouncelabs.com/securityexperts/

Security Training For ASP.NET Developers

17/18^th July at Leamington Spa. I have heard a lot about this course, and its all good. He has held similar talks at Black Hat and covers some really in depth topics of ASP.net security.

More information at http://www.nxtgenug.net/Course.aspx?CourseID=4.  Review of a previous event can be found at http://www.nxtgenug.net/Article.aspx?ArticleID=166

I would love to attend one of the courses, sadly I have no employer to pay for it (at the moment...)

I've been looking into Google Gears over the weekend, had other things on my mind so its taken longer than I wanted. My first thoughts are: pain.

Details of what Google Gears is can be found at http://gears.google.com/ with the developer reference at http://code.google.com/apis/gears.  Put simply, it saves a copy of all resources a developer references in a manifest and saves them in a local SQLite database on the client, with each site (protocol, url, port defines site) having there own directory and database for each browser it is viewed in allowing for isolation and improved security.  Still not 100% sure about this, I bet it will/can be hacked.  This database can also be used to store data, writing to the database from within javascript using standard SQL.  Today, I am interested in getting the basic Silverlight 1.0 template (clickable button) working with Gears in a offline state.

To start with, I installed Gears inside of my Orcas VPC as I have it preconfigured with everything and is going to be deleted when the next CTP is released (I'm hearing end of June). I then used the JavaScript on Google's developer site to enable the offline support, this has a number of problems which I will discuss.

As mentioned, Gears takes a copy of all the files required for the application to work offline. For this Silverlight project, the default is

Default.html
Default.html.js
Scene.xaml
Scene.xaml.js
Silverlight.js
gears_init.js (JavaScript for Google Gears)
go_offline.js (JavaScript for Google Gears)

These are all the files required to have the application running, in this case both online and offline.  You need to reference everything which the user might require, images, stylesheets etc.  If the user requests something not within the Google Gears database for the site, then it will first go to the browser cache, if it can't find it there it tries the server and fails if your offline.  This brings me onto an important point, with Google Gears it will always look in the local DB first even if your online and then check for updates/sync.

So lets break down how to setup the site to enable the files above to be downloaded.  My example is going to automatically save into Google Gears when the user first visits the site, in a real application it more likely that you will have a link which allows it to go offline.  This depends on a app-by-app basis.  Within Default.html, I reference the two JavaScript for Google Gears.

<script type="text/javascript" src="gears_init.js"></script>
<script type="text/javascript" src="go_offline.js"></script>

Gears init creates the ActiveX object, one way for each browser (IE, FireFox, Safari).  go_offline was taken from the tutorials on Google's site, but requires modifications.  This has all of the logic to create the database and save the files.  The local Google Gears deals with everything else.

With the two files reference, we need to call the correct functions.  Calling init(); will create the database and the store for the website. The createStore(); function deals with the saving of the items into the local database.  Within both files there is a check to make sure Gears is installed - if (!window.google || !google.gears) { //Not Installed }

Init() has two important lines of code.  The first line creates the database, giving it a name and the version number of Google Gears expected.  When first line is called, the dialog asking the user if the site is allowed to access Google Gears is displayed.  If the user clicks Deny, an exception is thrown so you will need to deal with this correctly within your application incase if ever happens.

localServer = google.gears.factory.create("beta.localserver","1.0");

The second line is creates the store for where everything is to be saved within the database.

store = localServer.createManagedStore(STORE_NAME);

Within createStore(); you first need to check to make sure store != undefined, as if the user clicked Deny it will be.  Two lines then give the manifest file for the URL, which I will discuss below, and then tell it to check for updates.

store.manifestUrl = MANIFEST_FILENAME;
store.checkForUpdate();

The manifest file is in json format, and tells Google Gears everything it should download in order to make the application work offline. There is then a version number which relates to the manifest, this is what Gears uses to decide if it needs to change its local list of files. If you do not increment, or the format is wrong, then Google Gears uses what it already has in the DB even if the file itself has changed.  This needs to be saved as Manifest.json an referenced to store.manifestUrl.

{
  "betaManifestVersion": 1,
  "version": "0.3",
  "entries": [
      { "url": "Default.html"},
      { "url": "Default.html.js"},
      { "url": "Scene.xaml"},
      { "url": "Scene.xaml.js"},
      { "url": "Silverlight.js"},
      { "url": "gears_init.js"},
      { "url": "go_offline.js"}
    ]
}

This caused a few problems, as at first I wasn't incrementing the version so any changes where not taken into account.  The second problem was that I forgot a comma, so the file wasn't downloaded but Gears did not make this known.

As mentioned, the main area is error handling, mainly making sure the user did not click deny and if they do handling the logic correctly.  The go_offline also has a textOut function to display some status information, however this throws an exception on the first execute.  I also found that after making a change to the web files, it took a while for Google Gears to update and take the change into account.  Not so bad for live sites, but makes development a lot more effort.  However, this could cause problems for sites like BBC News where users mainly check to see if there is new information, with Gears this could cause the user to see a older version without realising.

I also found having to disconnect from the network (in VPC change the network card to something else) and clear the cache a painful way to make sure the application was working correctly with Gears. Once I didn't have go_offline referenced in the manifest, it took a long time to actually realise, as sometimes it worked from the browser cache instead.  Might improve in later releases.  However, this does allow a offline Silverlight application which isn't a bad thing.

Download Sample

Demo @ http://blog.benhall.me.uk/Code/Silverlight/SilverlightGearedUp/Demo/Default.html

Code @ http://blog.benhall.me.uk/Code/Silverlight/SilverlightGearedUp/Code.zip

Labels: Silverlight

Yesterday I attended the Google developer day in London.  The event itself was cool, the event was held at The Brewery which was a very nice venue with open wifi for everyone,  this took a hammering but at least they offered it - unlike Microsoft UK.  When I arrived I was handed a Google swag bag, including T-Shirt, Goo (putty), sweets, bottle of water, promo stuff, mouse mat, and 256mb USB stick which was cool.  There was a 'Bloggers Lounge' which had loads of food, few PCs to use, cushions on the floor, tea, coffee, smoothies, everyone sitting around on there laptops and chatting. Very cool atmosphere. 

The keynote was very high level, gave a overview of what Google is doing for the developer and what they are doing in terms of mapping functionality. I then had the API workshop, which was to create a map mashup with Google guys on hand to help - was good but wish I had gone to talks instead. But the two talks I did go too, one of Google Gears and the other on mobile development where really really high level and didn't provide me with enough depths. The sessions where short, 45 mins, but they opened the floor up to 15 minutes of questions which meant there wasn't much in the way of content.  Would have been much better if anyone who wanted to ask questions grabbed the guys afterwards/in the bar as they where around. 

I do think Google missed an opportunity to demonstrate a lot of what they are currently offering and show how easy it is to integrate/use.  For example, the Google Gears presentation could have done 'Intro (5 minutes). Here is my Ajax app wrote using Google Web Toolkit, quick tour of it/code (15 minutes). Now lets put it on Gears - show how easy it is (15 minutes), job done.  Demo it all working (5 minutes)'.  Unless it's not that easy...

There was also an hour keynote streamed from the main Dev Day in the US, however we had all heard about this during the day and offered nothing new.  The code samples which where shown couldn't be made out which was a shame.

Afterwards,  Google had hired out a bar across the road with a free tab and food.  I had a few beers with Chris DiBona who is the open source program manager at Google which was cool and managed to get in a few feature requests.  Again, great atmosphere and decision by Google and was a good end to the day.

It was Google's first attempt at a large developer day.  The day/evening itself was great, and I hope they do one next year. Just a shame the sessions didn't match.  Could have just been the ones I went too. Will be interesting to see what HackDay will be like.