.NET From India

New Version of Enhanced Data Generator

I just uploaded a new version of the Enhanced Data Generator v 1.5. This does not have a new data generator, but rather bring the IBANGenerator back into the main DLL. Also I have now moved to SQL CE 4.0, which means you need not install SQL CE to run Enhanced Data Generator. All required SQL CE files now ship with the deployable.

More details at Enhanced Data Generator Home

Download From Enhanced Data Generator v1.5

Enhanced Data Generator v 1.4

I just uploaded a new version of the Enhanced Data Generators over the weekend. This new version 1.4 contains misc. bug fixes and the addition of a new data generator, XMLGenerator (Note: v1.3 and above support only Visual Studio 2010 and .NET 4.0).

The XML Generator is based on the XML Generator MSDN sample code and wraps it to work as a data generator. You basically provide an XSD and it generates XML that fits that XSD. Should be useful to fill XML type columns with valid data that fits a XSD.

Note: Since I use the MSDN sample, this does not support very complex XSD patterns. But I think, this should satisfy most of the common scenarios.

Download Enhanced Data Generator v1.4

Read More on Enhanced Data Generators

New version of Enhanced Data Generator

Just uploaded a new version of the Enhanced Data Generator, v 1.3.

Rather than new features this version is a move to .NET 4.0 and VS 2010. Since the internal VS classes that support Data Generators have changed in VS 2010, it is not possible to have backward compatibility. So from v 1.3 onwards, I will be supporting only VS 2010 and .NET 4.0.

Download the new version

Enhanced Data Generator 1.2.0

I have uploaded a new version of the Custom Data Generator Library I am building.

More information is available here and the Download is available here.

Major Changes are

• Seperated out the IBAN Generator to a seperate DLL so as to isolate the usage of the RegExString Class
• Added a new Generator to generate Indian Bank Codes
• Minor bug fixes

Enhanced Data Generators

I just finished creating a new version of the Enhanced Data Generators for use with Visual Studio Team System Database Edition (aka Data Dude), GDR R2.

This new version has the added ability to generate Chinese first and last names and also valid IBAN numbers. Also quite a bit of performance tweaks and bug fixes.

Instead of just writing blog posts about this, I have setup a mini website, with a couple of html pages that I can update and use for providing information about these components.

You can find the new website at

 http://www.dotnetindia.com/EnhancedDataGenerator/index.html

You can also find my older posts about these components at

 http://www.dotnetindia.com/enhanced-data-generator/

 

Please use the contact form to provide feedback about this component.

 

Technorati Tags: Enhanced Data Generators,Custom Data Generators,VSTS,GDR R2,Data Edition

The mysterious RegExString Class

As some of you may have noticed I have been writing a number of custom Data Generators for the VSTS Database Edition over the last couple of months.

During this time I have learnt quite a bit about the way these work and about writing these components. There is scarce documentation and I couldn't find too much information around writing these in the blogosphere too. Any information out there seems to be put out when VSTS Database edition was released and obviously a lot has changed since then with the release of multiple updates (with the latest being GDR R2).

One of the things that makes a lot of sense to me is to try and reuse some the Generators that Microsoft has written to create more complex Generators. One of the most interesting Generators is the RegEx Generator. I did quite a bit of searching around and am yet to find any code/freeware that does the equivalent and from a data generation perspective it is absolutely important to have the ability to generate random data based on a pattern.

The class that does this work is called RegExString. (I look at the Microsoft.Data.Schema.Generators.Extensions.xml found in the VSTSDB\Extensions directory will give you this information). If you search around for information on this class you will find a lone MSDN entry and most probably one of my forum posts.

If you just add the dll specified in the Microsoft.Data.Schema.Generators.Extensions.xml to your reference you will find no RegExString Class that you can use. A little bit of sleuthing will show you that the namespaces between the Dll and the MSDN entry are wrong. Well this is because, the namespaces have changed with GDR R2(I am not sure if it changed with R2 or GDR R1) and now all Generators are internal (or friend) and not public. So only Visual Studio can create instances of these classes.

Well now what about the MSDN documentation, which does not talk about the class being internal. Well that is for the older DLL, Microsoft.VisualStudio.TeamSystem.Data.Generators.dll. You should still be able to find this dll in the DBPRo\Extensions directory inside Microsoft Visual Studio 9.0. This is the older version where the Generators are still public.

So I decided to use those in one of the newer Generators I am writing (hope to get a new release out sometime later this week). I know this is a risk as most probably this is going to disappear when moving to VS 2010, but as of now, this is the easiest way for me to generate some pattern based data.

This Dll also requires a reference to the Microsoft.VisualStudio.TeamSystem.Data.dll, which can be found in the DBPro directory.

Once this is referenced you should be able to create an instance of the class. So I created an instance and took the class for a spin,

regx.Expression = exp
regx.MaximumLength = length
regx.GenerateNextValues()
value=regx.Result

Well this looks like this should work. If you try this you will get an "Object reference not set to an instance of an object" error. Searches and forum posts did not turn up any solution.

I had almost given up on this when I actually found the solution when playing around with the code searching for something else. The secret is to Initialize the Generator:

regx.Initialize(New GeneratorInit(conn))

This call just needs a valid connection object. This needs not be connected to a database.

So once the Generator is initialized, everything should work fine. This actually is true for some of the other inbuilt generators I tried too…

I really hope Microsoft makes the inbuilt Generators public again so that it is easier for people to develop custom generators and not reinvent the wheel every time.

Enhanced Data Generator – Bank Routing Number Generator

I have updated the custom Data Generator component I wrote sometime back. I have added a new generator to generate US Bank Routing Number aka ABA numbers.

This component can generate both the MICR format and the Fraction formats of the number. If you don’t know the difference then you can use the default which is the MICR format.

This can also generate the checksum digit correctly (so your checksum validation code can be tested) if required based on the GenerateChecksum property.

Note: Since the Fraction format does not support Checksum, it is automatically set to false when you change formats. But the IDE does not seem to correctly update this, so you may need to change the Checksum property twice after changing to the MICR format.

Just replace the DLL and related files with the new set of files and that is it. I have not built the setup kit this time.

Download DataGenerator v1.1.zip (1000.3K)

Note: Works with VS 2008 SP1 and Data Edition GDR R2

Technorati Tags: Visual Studio Data Edition,Custom Data Generators

Test Data Generation - EnhancedDataGenerator

One of the most problematic part of any new development is to make sure that you test your application with data that is near real life. 90% of the cases developers just test with a couple of rows of data that they enter and the data is rarely diverse enough or similar to real life data.

There are a number of data generator tools out there in the market, but they are costly and development teams rarely get the budget to buy these tools. There are a number of random data generation free-wares, but most of these are developed over a weekend hacks and though they generate random data, I don’t think they are representative of most real life data (Who actually has a name like “$ssoghetn2u#a”).

Few people know that Visual Studio Database Edition has a set of tools to generate test data. Called Data Generators, you can add a data generation plan to your database project and use that to generate test data. It ships with multiple data generators and you can also write custom data generators very easily. Since the Database Edition is now available with the developer and some of the other editions, it is easy to use by most developers.

I actually started writing a data generation application that can generate real life representative data a couple of years back. As is the fate of a number of those weekend projects, this went into hibernation once I ran into time issues and then was forgotten.

Then a couple of months back I started playing around with the Visual Studio Team System (VSTS) Database Edition and then remembered I had some code that I had written for the original data generator and so started looking into writing custom data generators for VSTS DB. There is not too much data on writing custom generators and most of the blogs are out of date. So if you want to write custom generators, I would point you to MSDN.(That information is quite good and you should be able to get up and running quickly)

I was surprised at how the VSTS DB custom generator interface was so similar to what I had come up with and so started writing a couple of data generators based on the code I already had. After a couple of weekends, I had my first couple of custom data generators.

Note: These are written for VS 2008 SP1 and Database Edition GDR R2. I am not sure if this will work for GDR 1. (I suspect it will as the interface for custom data generators have not changed).

This EnhancedDataGenerator currently contains two DataGenerators:

1. FirstNameGenerator – Generates real-life First Names

2. LastNameGenerator – Generates real-life Last Names

These are also locale aware and currently support two locales en-US and de-DE. So based on the locale this generates different kinds of First and Last Names. (Locale is a property in each generator, so you can generate English first names and German last names, if that is what you want). If you have data for common First and Last names in other cultures, please send them to me and I will update the database to handle those cultures too. (The maximum time I actually spent was to search for and get the base data into a format that I can import into the database)

The FirstName Generator is also Gender aware.

A couple of screenshots of these at work:

en-US	de-DE

EnhancedDataGenerator – DLL and config Files. For people who know how to do a manual deploy of custom extensions.

EnhancedDataGenerator Setup – Setup File that will deploy the files to the default VS install location (Don't use if you had used a custom path to install Visual Studio)

Other Things to Notes

1. Error Handling is not great. I have not yet found a way to report errors to the IDE user. So as of now if there are any errors, nothing happens. E.g. if you set the locale to anything other than en-US or de-DE, you just get empty data back. So if no data appears, that means you have an error .

2. The setup kit has not be fully tested. So if this does not work for you, use the DLL kit and do a manual deploy

3. Retain the directory structure inside the zip file when doing manual deploy

3. If your Visual Studio is installed to a custom path (not <prog Files x86>\Microsoft Visual Studio 9.0, then you will have do a couple of more things.

a. Use the manual depoy method and get the Extensions deployed in the VSTSDB\Extensions

b. Change the config file setting DBPath to point to the Database directory inside your custom extensions directory.

4. The .Extensions.xml file is already created with the correct data, so you will not have to create those again when doing a manual deploy.