This article talks about how you can use My developerWorks to personalize your view of developerWorks content and connect and collaborate with your trusted peers

Learn about the new My developerWorks professional network and knowledge base in this overview demo

My developerWorks is a new professional network and knowledge base connecting the technical community worldwide

PostgreSQL 8.4 has come out, and while I am a bit disappointed that PostGIS 1.4 has not come out for fear that we've missed a bit of the PostgreSQL 8.4 momentum, I am happy that we are nearing closer and just maybe we'll have it out by end next week. We now have a PostGIS 1.4RC1 http://postgis.refractions.net/download/postgis-1.4.0rc1.tar.gz tar ball as well as experimental binary builds of this for windows user's running PostgreSQL 8.3 http://postgis.refractions.net/download/windows/pg83/experimental/postgis/ or PostgreSQL 8.4 http://postgis.refractions.net/download/windows/pg84/experimental/postgis/. Please give both a try.

Working in the Cathedral Really?

As Paul duly noted in his blog entry Working in the Cathedral the model for PostGIS development is morphing, but I wouldn't call this morphing process one that is entirely toward the Cathedral model. Unlike the perceived Cathedral model, I would like to think we will have more frequent releases and beta releases, perhaps parallel experimental builds and most importantly, more fun. The main idea being making it much easier for mere mortals and fake mortals to taste test the cookies in the oven while they are cooking. By fake I mean unit tests, build bots, and computer generated people where the fear of destruction is removed. I feel this is the similar model PostgreSQL goes by or is trying to achieve.

Continue reading "PostGIS 1.4 hot on the heels of PostgreSQL 8.4"

In a MapQuest-like display of flat-footedness, the original URL-shortener TinyURL seems on the cusp of being crushed by the new kid in town, Bit.ly.



To me, it happened overnight. One day everyone was using TinyURL on Twitter, the next Bitly. As it turns out, that wasn't an accident as I learned in this TechCrunch story, URL Shortening Wars: Twitter Ditches TinyURL for Bitly. It seems that Summize (subsequently acquired by Twitter and now Twitter Search) and Bitly are backed by the same entity, Betaworks, which also has common investors with Twitter.

So my gut says the story goes something like:

• Hey, we're driving a lot of traffic for TinyURL
• Maybe we should get that traffic ourselves
• I bet we could get Twitter to make us the default URL shortener
• And -- this part's also key -- I bet we could do it better

Hence I wasn't surprised to start reading stories in the past two days about Bitly's big vision. See, for example, this TechCrunch story, Bitly's Grand Plans, and Their Inevitable Clash with Digg: Bitly Now.

Excerpt:

The magic behind Bit.ly are the stats that the service makes available on the underlying domains being clicked. Investor John Borthwick explained it all to investors in an email we obtained earlier this month:

bit.ly has been on a tear since we launched it last summer ...bit.ly is on its surface a link or URL shortener, helping people take long and unwieldy links and make them short and easy to share via email, Twitter, Facebook etc. But once you shorten a link with bit.ly the fun begins. You can put a simple “+” on the end of any bit.ly link and see, real time, the pace at which that link is getting shared and clicked on as it moves around these social distribution networks.

Bit.ly Now will take all of this deep (and wide) data on popular real time URLs and turn it into a service. That’s where the inevitable clash with Digg comes in.

Bitly, as it turns out, think it has some key advantages against Digg, reminding me that the web is also not for those bad at math.

Bit.ly says that the data flow they are seeing is so massive that they are getting very good at predicting the number of clicks a link will get in the future. They look at acceleration of clicks as well as the source (Facebook, Twitter, IM, whatever) and whether people are clicking that are outside of the social graphs of other people clicking.

In other words, you could say that Bit.ly knows what will be on the Digg home page tomorrow.

The amazing thing, from a strategic marketing perspective, is that TinyURL has been written out of the story. By looking ahead towards Digg as the new competition, by painting a vision of where it wants to go, by discussing how it wants to get there, Bitly just blows by TinyURL and writes them out of the story line. This isn't about URL shortening; it's about information sharing and communications.

Well done! I sometimes call this Lot's Wife's Law of Marketing Communications: don't look back; only look ahead.

The lesson for TinyURL is that you can't remain static, or someone will come along, reinvent you with a broader vision, and paint you out of the picture -- turning you, if you will, into the pillar of salt.

I was trying to demonstrate to a client how to monitor queries that generate internal temporary tables. With an EXPLAIN plan you see ‘Creating temporary’. Within MySQL you can use the SHOW STATUS to look at queries that create temporary tables.

There is the issue that the act of monitoring impacts the results, SHOW STATUS actually creates a temporary table. You can see in this example.

mysql> select version();
+-----------------+
| version()       |
+-----------------+
| 5.1.31-1ubuntu2 |
+-----------------+
1 row in set (0.00 sec)

mysql> show global status like 'created_tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 48    |
| Created_tmp_files       | 5     |
| Created_tmp_tables      | 155   |
+-------------------------+-------+
3 rows in set (0.00 sec)

mysql> show global status like 'created_tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 48    |
| Created_tmp_files       | 5     |
| Created_tmp_tables      | 156   |
+-------------------------+-------+
3 rows in set (0.00 sec)

What has perplexed me in the past, and I can’t explain is that SHOW SESSION STATUS for this example does not increment. It’s confusing to tell a client to use SHOW SESSION STATUS for SQL statements, but the behavior is different with SHOW GLOBAL STATUS. For example, no increment.

mysql> show session status like 'created_tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 0     |
| Created_tmp_files       | 5     |
| Created_tmp_tables      | 2     |
+-------------------------+-------+
3 rows in set (0.00 sec)

mysql> show session status like 'created_tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 0     |
| Created_tmp_files       | 5     |
| Created_tmp_tables      | 2     |
+-------------------------+-------+
3 rows in set (0.00 sec)

Let’s look at a query that creates a temporary table.

mysql> explain select t1.* from t1,t2 where t1.c1 = t2.c2 order by t2.c2, t1.c1;
+----+-------------+-------+------+---------------+------+---------+------+------+---------------------------------+
| id | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                           |
+----+-------------+-------+------+---------------+------+---------+------+------+---------------------------------+
|  1 | SIMPLE      | t1    | ALL  | NULL          | NULL | NULL    | NULL |    3 | Using temporary; Using filesort |
|  1 | SIMPLE      | t2    | ALL  | NULL          | NULL | NULL    | NULL |    3 | Using where; Using join buffer  |
+----+-------------+-------+------+---------------+------+---------+------+------+---------------------------------+
2 rows in set (0.03 sec)

If we use session status we get an increment of 1.

mysql> show session status like 'created_tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 0     |
| Created_tmp_files       | 5     |
| Created_tmp_tables      | 2     |
+-------------------------+-------+
3 rows in set (0.00 sec)

mysql> show session status like 'created_tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 0     |
| Created_tmp_files       | 5     |
| Created_tmp_tables      | 2     |
+-------------------------+-------+
3 rows in set (0.00 sec)

mysql> select SQL_NO_CACHE t1.* from t1,t2 where t1.c1 = t2.c2 order by t2.c2, t1.c1;
Empty set (0.00 sec)

mysql> show session status like 'created_tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 0     |
| Created_tmp_files       | 5     |
| Created_tmp_tables      | 3     |
+-------------------------+-------+
3 rows in set (0.00 sec)

If we use global status, in this case it’s and idle server so I know there is no other activity, however in a real world situation that isn’t possible.

mysql> show global status like 'created_tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 48    |
| Created_tmp_files       | 5     |
| Created_tmp_tables      | 171   |
+-------------------------+-------+
3 rows in set (0.00 sec)

mysql> select SQL_NO_CACHE t1.* from t1,t2 where t1.c1 = t2.c2 order by t2.c2, t1.c1;
Empty set (0.00 sec)

mysql> show global status like 'created_tmp%';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 48    |
| Created_tmp_files       | 5     |
| Created_tmp_tables      | 173   |
+-------------------------+-------+
3 rows in set (0.00 sec)

Read the Declaration of Independence and then it’s time for fireworks and grilling, maybe some apple pie too!

Greetings from Zurich.

The BEST Pattern Award is....

Pattern: Query Visitor (download .PDF)
Author: Adrian Marriott,Principal Consultant, Progress Software Inc.

2nd Place:
Pattern: Schema Builder (download .PDF)
Author: Richard Lingeh,Principal Consultant, Versant.

3rd Place:
Pattern: Bespoke Indexes (download .PDF)
Author: Adrian Marriott,Principal Consultant, Progress Software Inc.

The Awards ceremony took place on July 2, 2009, at the ICOODB 2009 conference in Zurich, during the evening reception.

All 25 persistent patterns submitted are available for free download (LINK).

The DB2 folk have published a few performance tips for you to aid you as you upgrade to DB2 9 for z/OS. It's short, sweet, and to the point. You'll want to check it out to make sure you are taking advantage of the suggestions being made."DB2 9 migration: SQL performance best practices"

All licenses are 10% off for the duration of the sale.  If you contact us after the sale, sorry, you missed it.

You want to the source code added on to your Small Business or Corporate account we are offering it for 50% off these same two days.  Open a ticket with your information request and we will take care of you.

 Visit the online pricing page for more information.

No, this is not a pricing change.  This is a short and sweet 4th of July Fireworks sale.

Rolling Thunder ($7.99 @ Amazon) by John Varley is the third book in the series. I previously reviewed Red Thunder and Red Lightening, having really enjoyed them both. In this third book the prime character is now the 3rd generation (or 4th depending on how you count I guess!) and is told from the perspective of Podkayne (Elizabeth), a singer and musician in the Martian Navy. She ends up on Europa (Jupiter moon) when an interesting and major event occurs with severe repercussions on Earth, and which makes her the most famous person in the universe.

I didn’t enjoy the book as much as the first two. Maybe part of that was that at times the story/character is pretty shallow – she’s 19/20 at the time the story starts and that’s not meant as sexist (though maybe it is), just that few 20 year olds think really deep thoughts. Parts of it just come together in a way that feels forced in an effort to arrive at the ending.

Though I didn’t enjoy it as much, I still like the series being told from different generational perspectives. Think that is a nice device and a nice change of pace.

I've had a number of people tell me over the years that they aren't surprised when I say "Recommended" at the end of any book review I write, as I never seem to say "Not Recommended". I suppose that comes under the heading of "if you can't say anything...(read more)

If you're working with SQL Server, chances are pretty high that you already realise that your future involves SharePoint as well. Fellow Australian MVP Ishai Sagai recently sent me a copy of his new book SharePoint 2007 How-To . I have to say I really...(read more)

At Percona we provide services both Onsite - visiting the customers and Remote - logging in to their systems or communicating via email,phone,instant messaging. We believe both approaches have their benefits and drawbacks and mixing them right way allows you to get your problems solved most efficient way. Onsite visits are great as they [...]

The Scaling with Flash webinar I’ve mentioned earlier was a success and we got the recording available. It contains Percona presentation, presentation of Schooner appliances and Q&A session. Enjoy. Entry posted by peter | [...]

I have been working on this blog post for around six months.  I keep writing and shelving it.  Today I read a blog post from the DexExpress CTO that got me back to wanting to publish this post.

Early Adopters?

To put what Julian said in different terms; Do you want all the Early Adopters for your product?   There are pro and con arguments to having the early adopters when it comes to technology. 

While they tend to be some of the most passionate people out there, and they are usually eager to play with a technology but have little to no loyalty to that technology.  By the time you have stabilized your product cycle they will be long gone on to the next technology buzzword to play with it.

They also tend to have the most harsh reviews of the early technology.  They will bash it as not complete, or not like product x, etc.  But by the time you have addressed their issues they will never come back to update those huge blog rants they did against you in the early days.  Their social media contracts tend to be very one sided – whatever is in it for them today.  After all anyone on the Internet can post a blog slamming a product or technology as inferior.  You don’t need credentials or facts to back you up.  Just post it – people will believe it.  And if the company actually listens to you and implements your suggestions you can always shrug them off as not being “agile” enough for you (IE, they didn’t give you that build the same day you asked for it).

Buzz word and gone

Microsoft is great at playing into this early adopter hype.  They print magazines with all their buzz word laden technologies sometimes years before you or I will be able to put it into production.  And by the time you do get around to wanting to use it in production they have often already abandoned it for another, newer, faster, more buzz word laden system.

Look at Windows Workflow (WF).  Anyone who early adopted it is probably kicking themselves now.  Microsoft has basically punched reset on the entire design.  I really looked at implementing VistaDB serializers for WF.

They can’t remove it from the framework!

And to all the people who use the argument that they can’t remove it from the framework - that is just silly.  LINQ 2 SQL (L2S) is a good example of this.  Microsoft printed tons of material about it, but you weren’t supposed to take it seriously.  It was just until the more serious Entity Framework came along, didn’t you know? 

Well, from the number of articles, books, etc I don’t think anyone understood that argument.  So if you did buy into that technology what about .Net 4?  Well, Microsoft has said that L2S will exist in some capacity, but what does that mean?  To me it means it means you had better be embracing .Net 3.5 SP1 for a very long time. 

If your favorite buzzword technology was implemented in a SP how stable is that to actually deploy?  I know lots of shops that still refuse to deploy .NET 3.5 SP1 because of the problems they had with the “Service Pack” changing how things worked for their applications.

Sync Services

We actually got bit by Sync Services rewrites three times before we stopped.  Is it stable today?  I don’t think so. From what I have read on the renamed and polished Sync Framework (as it is now being called) it is still being tweaked and changed.  There are lots of people pushing for changes for it to support Azure (ahem, aka cloud computing, aka SQL Server in the Cloud, whatever they want to call it today).

Got Entity Framework?

I have gotten a number of emails from users wanting to look at our Entity Framework (EF) provider.  After all didn’t we talk about it late last year?  Yes, and we did implement it.  The entire VistaDB EF provider is complete.  But we will not ship it until we know it performs good enough, and that it will be around for us to support our customers.

Ship it!

So why not ship it?  Why not give it to all those people who want it?  Why not ride that Early Adopter wave? 

Basically, because I won’t do that to you.  I actually care about the technologies and interfaces we unleash on customers.  I want to ensure whatever technology we present publicly as an interface is stable and not going to go away on you without warning. 

We have been making lots of changes in the engine to support patterns that are particular only to the EF provider.  They are not complete, and may not be by the time we release 4.0.  There are just a lot of strange behaviors in the way it works because it is LINQ based – not relational based.  We want to ensure that the majority of the common patterns are handled with decent performance, and then continue to improve the engine over the lifetime of 4.x.

What type of mismatch?

All ORMs exhibit a mismatch when they are mapped onto relational data.  It is known as the Relational Impedance Mismatch and all ORMs have it to some extent.

An example of the problem in real world terms:  If you have a group of objects you might like to foreach() over them to do something. With Entity Framework that might result in thousands of round trips to your database to answer each of those foreach iterations with one single query.  You are much better off to work on a result set and pair things using set operators.   EF is supposed to provide for this scenario through DataSets, but it is actually highly dependent on what code the programmer actually wrote.  If the programmer does not understand what the framework is going to generate they can beat the crap out of database connections.

We have found some common CLR coding patterns that translate horribly to EF generated database calls.  We have to address these issues before we ship something.

Support and Documentation Problems

Another problem with EF is the documentation.  We are an ADO.NET compliant database engine.  That means if you understand ADO.NET patterns and practices you can pick up our engine very easily.  There is no such level of competency with EF.  Everyone will be in a huge learning curve.  Most of the documentation you find for LINQ is around Linq2SQL syntax (which is NOT the same).  And to make it worse that L2S syntax will compile and blow up and runtime in most cases with very bizarre runtime errors.  This would cause a huge support load on us.  I cannot imagine how we would be able to keep up with those types of tickets when most of the issues have nothing to do with us, but the EF framework itself.

So if we ship an EF provider we are in effect having to support the entire EF framework including tools designers and runtime.  That is a REALLY tall order for a small company. 

Entity Framework Stability

So if we are going to have to support it, how stable is it?  In my experience of our online store and related tools it has a ways to go for stable.  I still have to regen our store edml model about once a month because Visual Studio somehow corrupted it or got out of sync.  I also still have the GUI wizard in Visual Studio crash on me regularly.  And this is all when working with SQL Server. 

Who do you think is going to get those tickets if they were working with our EF Provider?  We would, and there is nothing we can do to fix them.

I know of five companies that started using EF last fall for major projects.  Out of those five only one is still using it in any way.  All of them abandoned it for their main DAL, but one was still using it for internal tools and  utilities.  That tells me the framework has a ways to go for widespread adoption.

When will we ship it?

The current plan (and this is of course subject to change) is to start preview builds of 4.0 shortly after we ship 3.6.  VistaDB 3.6 is feature complete at this point.  We are just working on the samples and fine tuning the installer.

Rather than release VistaDB 3.7 we are going to start releasing VistaDB 4 preview builds.  I will post more information on what that means in engine terms later, but our plans are to release a series of preview builds and then ship a stable around the October timeframe. 

There are two things that have to happen for us to ship an EF provider. 

One is that I have to see the .Net 4 EF model and know we are not going to be required to totally rework our existing code.  If we have to rewrite then I will never release the current provider.  It would be a waste of time to ship something that is obsolete almost as soon as it is released. I want to make sure we can support the VistaDB EF Provider for a minimum of 2 years.

The second is the Visual Studio 2010 plugin model specifications.  Right now our current plugins do not work with VS 2010.  If Microsoft is going to leave the current model in place and require us to update the designers then we will have to create a new build for VS 2010. 

Visual Studio Support

We cannot afford to support three versions of Visual Studio (each with it’s own designer) at the same time, so it would mean a drop of VS 2005 support for VistaDB 4.  I don’t want to do that.  I am hoping that Microsoft fixes their current issues, or publishes a way for third parties to not have to build multiple plugins to support different Visual Studio editions. 

I don’t think that Visual Studio 2010 is going to be a runaway adoption from day one.  I find that a lot of companies are just now migrating to Visual Studio 2008, but many are sticking with 2005 and hoping to leapfrog directly to 2010.  But it comes back to that Early Adopter statement above.  Those users who adopt Visual Studio 2010 early are going to be vocal users.

.NET 3.5 SP1 or 4 support?

You can’t build an EF provider without using .Net 3.5 SP1 – it simply didn’t exist before that time.  That means you are dependant on a SP being present.  (WHY, oh WHY didn’t they call it 3.6 since it added so many new features!)  This means we have to ship a .Net 3.5 SP1 assembly to support EF.  Would you do that and commit to that SP for the next 2 years, or wait for .Net 4?

There are a lot of great things in the .Net 4 framework.  We would love to take advantage of them, but you just can’t do that without also maintaining .Net 2 versions at this time.  I do eventually think that .Net 4 will replace the current .Net 2 stable that most people use.  I think a lot of companies are going to jump straight to 4 from 2 when they finally switch their compile targets. And that means the version of Entity Framework in .Net 4 is a very serious factor for us in what to support in VistaDB.

Right now I think we are leaning towards 3.x staying a .NET 2 assembly without Entity Framework Support.  Then stepping up to .NET 3.5 SP1 for VistaDB 4 with EF.  That would mean no Visual Studio 2010 support in the initial releases unless Microsoft changes something.

What are you doing?

What about you? What are you doing with regards to .Net versions and Visual Studio?  Are you going to start shipping 3.5 SP1 tied assemblies?  Or would you rather wait for 4? 

Is your company still shipping 2.0 assemblies today and skipping .NET 3 entirely?

I hope this has provided a little glimpse into how complicated it is for us to decide how and when to ship things.  It is never easy, but the entire .Net 3 product cycle has thrown a lot of our plans into a spin.

Just a quick post to highlight and share this great case study by Mitch Kramer of the Patricia Seybold Group on Blue Guru, JetBlue's content management and publishing system.

Excerpt to tempt you into reading the 26-page document:
XML is BlueGuru’s enabling technology, and MarkLogic Server is its most critical architectural element. XML addresses JetBlue’s requirements for structured documents—multiple types, multiple components within each type, hierarchical relationships between components, and component sharing across documents. MarkLogic Server is an XML content management system that automates BlueGuru’s documentation processes. Its repository stores BlueGuru’s documents and supports their access and retrieval by Crewmembers, partners, and regulators.

This case study report tells the story of JetBlue’s business transformation from a documentation system of decentralized and manually maintained manuals to a distributed content management and publishing system.I've embedded the full document below in Scribd epaper format. Thanks to Mitch for writing a great document and to the folks at JetBlue for their faith in us, for their support of Mitch in writing the case study, and for the help and input they've provided us.

I arrived in Richmond, VA this morning earlier than expected. I was meeting a few people and things were pushed back, so I had the chance to go by Best Buy and find a USB->VGA adapter. Apparently they don’t have the HP VGA cable, so I got this adapter instead.

I opened the box to find the device, and a CD. That’s interesting because the netbook doesn’t have a CD drive. I’d debated about grabbing an external DVD drive when I was in the store, but my bag is heavy enough. With my laptop, power adapter, extra USB disk drive, and now the netbook it feels like I’m carrying quite a load.

So how to get this working? Luckily I had an idea. I hit the local Starbucks and then connected to the web, downloading drivers from Tritton Technologies. One annoying thing was once I’d gotten to the site, they list a number of models, all of which look alike. I had the device with me, but not the box, and there as no model number printed on the device. A serial number, but that didn’t help. Grrrrr.

I had to go back to the car, look at the box, and then download the driver. I suspected that there was one driver for all models, but I hate trying to uninstall drivers. I’m still a little gunshy after all these years working with Windows.

Once that was done, I wrote a couple blogs, including this blog on the device. Some impressions:

• It’s small. Carrying it one handed is nice and easy. It’s much, much smaller than the laptop, and weighs only slightly more than the Kindle. That is nice.
• It doesn’t work great in the car. I had the box with me and tried to prop it on my lap to download drivers, but it was too small. Had to come back inside to a slightly bigger space to easily use it.
• One the flip side, you can pretty easily hold it up with one hand and use the other to work the mouse.
• I use page up/down, home, and end keys extensively. These require the Function key to work (they are on the arrow keys), so that takes a little getting used to.
• The keyboard works well for me. I type pretty quickly, and it works well for me.
• The CPU is slow. There are times you click something and it seems to take a minute to switch over. Something to be aware of as you install things.
• I have Firefox, Twirl, LiveWriter running, seems to work OK, other than some slowness to switch apps.

I’ll have more impressions over the next few weeks as I use this and compare it with my laptop.

With the current environment of data sensitivity, there is a need to mask data not only in the traditional relational databases, but also in flat files that could be part of a testing scenario. Customers continue to need to send files to vendors, off-shore testers, and even internal testers, and the data in those files could be subject to regulations. In this article, learn how to de-identify private data in flat files by using IBM InfoSphere Federation Server to create a relational table (nickname) for data in flat files and then applying data privacy algorithms using IBM Optim Data Privacy Solution.

IBM(R) InfoSphere(R) Data Architect (IDA) (formerly Rational(R) Data Architect) is gaining momentum as a comprehensive tool that helps organizations promote a thorough understanding of their enterprise information architecture. As more people use IDA, there’s an increasing need for some customers to extend IDA to meet their unique data modeling and integration requirements. This two-part series shows you how to extend IDA’s models, properties view, model reports and validation rules. In Part 1, you learned how to programmatically traverse and modify IDA models and how to add and display custom properties. In Part 2, learn how to generate customized reports and how to add your own validation rules for IDA models. [2009 Jul 02: This article was updated to reflect that the GUI for BIRT report customization has been changed for performance improvement. This is shown in the section that describes how to generate customized model reports using BIRT.--Ed.]