Delphi Win32 gets a TStringBuilder class

The next version of Delphi, code named “Tiburon”, is getting some cool new features.  One of these will be the TStringBuilder class and Andreano Lanusse (a CodeGeart Evangelist Leader) blogged about it.  Basically, it’s a Delphi implementation of the .NET StringBuilder class.  Only it’s available in the managed and unmanaged versions of Tiburon.  Why is this cool?  If you have code that does a lot of string manipulation, using the string class gets expensive, performance-wise.  If you are building up a string piece by piece, each time you add or modify a string, the previous version of the string is discarded and a new string is created.

The StringBuilder class pre-allocates the memory so that you avoid the constant creating and freeing up of string objects.  If you have code running in aloop, you can get real performance benefits by using StringBuilder.  I’m looking forward to getting Tiburon when it’s released.

How to shoot yourself in the foot with regular expressions

I had some Delphi code that is used a user entered text string as a key. After the user entered in the text, my code would force it to uppercase and strip out any characters outside of A-Z and 0-9. This was for a very specialized task and the following Delphi code did the job.

function GetStrippedValue(const value: string): string;
var
i: integer;
begin
result := '';

// strip punctuation out of the name and force to uppercase
for i := 1 to length(value) do
if value[i] in ['A'..'Z', 'a'..'z', '0'..'9'] then
result := result + Uppercase(value[i]);
end;

Nothing elegant, and performance wasn’t an issue as it was rarely used. I wrote that code in the spring of 2000 and it hasn’t been touched much since then. I now have a C# application that needs to work with the same data, so I to perform the same functionality using C#. Since C# does not have the set operators that Delphi has, I had to strip out the characters in a different way. I figured that I could use a simple regular expression and match the characters that way. I should have remembered that old quote attributed to Jamie Zawinski:

Some people, when confronted with a problem, think “I know, I’ll use regular expressions.”

Now they have two problems.

I had decided to make the expression as simple as possible. I used the following function to accomplish what I had done years earlier in Delphi:

private string GetStrippedValue(string value)
{
string pattern = @"[^\w]";

string result = Regex.Replace(value.Trim().ToUpper(), pattern, "");

return result;
}

What I am doing here is to replace any text that matches the Regex expression with an empty string. The “^” caret character negates the expression, in other words replace anything that doesn’t match the “\w” token with an empty string. I had found the “\w” documented to match any non-word character, but I didn’t look too closely to the definition. I was using the syntax defined at www.regular-expressions.info, where it is documented which matches letters ,digits and whitespace. That turned out to be different from the definition used by .NET and Python.

With .NET, that pattern allows alphanumeric characters, PLUS the ”_” underscore character. A slight, but fatal flaw on my part. I didn’t catch it when I wrote because my sample data didn’t have any underscores in them. I found it today while adapting that code for a new task, where the data can have underscores.

Einstein put it best, “Everything should be made as simple as possible, but not simpler.”. My attempt to write the least amount of code was flawed. What I should have done was to use the following syntax:

private string GetStrippedValue(string value)
{
string pattern = @"(?i)[^A-Z0-9]";

string result = Regex.Replace(value.Trim().ToUpper(), pattern, "");

return result;
}

This way, I am explicitly defining the allowable characters and now I’m getting the results that I wanted. Since I am passing in a string that is being forced to uppercase, I don’t need to test for lowercase letters (“a-z”). An alternative version that produces the same results would be:

private string GetStrippedValue(string value)
{
string pattern = @"[^A-Z0-9]";

string result = Regex.Replace(value.Trim().ToUpper(), pattern, "",System.Text.RegularExpressions.RegexOptions.IgnoreCase);

return result;
}

I’m not sure if one way is better than the other. The “(?i)” is equivalent to the System.Text.RegularExpressions.RegexOptions.IgnoreCase RegexOption. For this project the performance of the code is not an issue, it only gets called a few times.

The moral of this story is if you need to use regular expressions, check the syntax for the implementation that you are using.

Black Crayons and the school psychologist, a true story (just not mine)

I follow Raymond Chen’s blog (required reading if you program for Win32) and the other day he had a link to a story that was both hysterical and scary.  My first thought that it was another urban legend.  After a bit of searching, I tracked down the author, Deirdre Sholto-Douglas, and she confirmed that the story was true and gave me permission to repost it here.

It had been originally posted in the alt.peeves newsgroup on USENET back in 1994.  Click here for a reasonable definition of alt.peeves or here to access that group. I’ve included the text as posted, with the exception of the email addresses.  It was posted before the spammers started harvesting email addresses out of USENET and I see no reason to make these addresses public.


Forwarded-by: bostic@CS… (Keith Bostic)
Forwarded-by: good@p…. (Craig Good)
From: finch@M… (Deirdre Sholto-Douglas)
Newsgroups: alt.peeves

W. Blair Haworth Jr. (bhaworth@a…) wrote:
: In article 331ipk$omb@v.’.. finch@… (Deirdre
: Sholto-Douglas) writes:

: >You go ahead with your kidlessness and while you do so, I’ll continue to
: >kiss skinned knees, assemble bicycles, and enthusiastically accept popsicle
: >stick ‘ashtrays’. 

: Ashtrays!  Godamightydamn, things being what they are these days, I’d’ve
: figured that any kid that turned out an ashtray in arts’n’crafts would be
: immediately sent to Special Ed. for re-grooving and Social Services
: notified to investigate the home environment. 

Your comment reminds me of one of the run ins I’ve had with the school psychologist.  After our disscussion, I’ve gained a reputation of being unreasonable and my daughter has gained the freedom to construct anything short of atomic bombs without psychological interference.

If I haven’t peeved about this in the past, I certainly should have and if I have, I beg the readers’ indulgence.

It all started shortly after my ill-fated Parent-Teacher Conference (as did my public school reputation for being unreasonable).  Within a week of this conference, I received a phone call from said ‘psychologist’  requesting that I present myself in his office to discuss my daughter’s ‘problem’.  When questioned, he indicated that the ‘problem’ was different than the one the teacher and I discussed (which was not addressing adults by first name), but coyly refused to ‘discuss a situtation of this magnitude over the phone’.

The following day, at the appointed time, I appeared with offspring in tow. Horrified looks resulted and said offspring was shuttled off to play in the gym.  Apparently these discussions are SECRET.

He began by folding his hands on top of his desk and wearing his ‘saintly, patient’ expression.  *This* is a man who has not only READ the psych books but *believes* them.

“Has Lauren appeared depressed or been behaving unusually at home?”

“No, she has not.”

“Her behavior *hasn’t* changed?!”

“No it hasn’t.  Pardon my abruptness, but precisely *what* are you driving at?”

He is now refusing to meet my eyes and fiddling with a paperclip on the desk.  Hmmm.  I should have trundled my copy of ‘Body Language’ along with me.  He could have fidgeted and I could have merrily looked up all the  underlying pyschological causes. 

“Well, erm…you see, Lauren is using only black crayon when she’s drawing and studies have indicated that when this occurs the child is usually depressed and attempting to deal with repressed emotions.”

“Ah.” 

At this point, I was having considerable difficulty repressing one of my own emotions….namely laughter.  What rocks do these nitwits crawl out from under?  Realising that my original response would be a Bad Thing, I quickly pasted my Concerned, But Amused Parental Expression on and continued:

“Have you considered asking Lauren her reasons for using black crayon?”

Shock.  Horror.  Complete dismay.  He actually began stammering.  One does NOT ask the child.  It could cause deep-seated emotional problems, stunt their growth, cause them to suffer from low self-esteem and possibly begin hanging about on street corners with gangs of second graders. 

I excused myself from his office, obstensibly to collect myself, in actuality to collect my offspring from the gym.  I arrived at said gym to find my depressed, repressed, emotionally devasted monster attempting to deal with her deep-seated frustration at not being able to reach the rings.  Was she crying, fussing or sulking?  Nope.  She was trying to negotiate with the custodialdrone for a stepladder.  At this point, I decided she was entitled to draw with black crayons the rest of her life, if that’s what she wanted.

We meandered back to the office and I ignored the look of distress that was shot at me.  I parked my recombinant DNA in a chair with orders to ‘Behave like a lady.’ (Yeah, I know.  So sue me.)  The conference resumed, this time I addressed my questions to Lauren.

“Lauren, Mr. Shit-for-Brains indicates that you only use blackcrayon when you’re drawing.”

“Yeah.”

“Do you *like* drawing in black?”

“No.”

“Then why do you do it?”

I was treated to the expression that is reserved for humouring slightly thick parents and watched as my offspring pasted on her Mom’s Old Lady But Harmless Expression:

“They make us line up in alphabetical order when they pass out the crayons.  And I’m always last in line…there’s nothing left *but* black!”

I turned to witness what our psychological brainchild is making of all this.  He has gone strangely quiet.  Fine.  This interview is over as far as I’m concerned. Although I confess, I couldn’t resist lobbing one more over the fence at him.

“Thank you sooooo much for your concern regarding my daughter’s emotional well-being.  I suppose your job would be *much* easier if all depressions could be cured by simply starting the crayon box from the other end of the queue.  In the future however, do you think you could at least ask *her* before you haul me in here?”

He managed to mutter something which I took for assent neither Lauren or myself has heard anything from him since.

ObPeeve:  School Psychologists that are looking for deeper meanings in simple kids’ actions.

!Peeve:   He now scuttles around the nearest corner whenever he sees me coming.  I think the poor man probably suffers from deep-seated, repressed emotions.


Playing with a new toy, a GPS device

While on vacation in Hyannis, I got annoyed with the one way streets odd layouts so I went out and bought a GPS unit.  I had wanted one for a while, and this was a first good excuse to get one.  The Hyannis Staples had a Magellan RoadMate 1412 on sale, so I bought one.  Staples had it for $250, but you can get it cheaper online.  It’s your basic consumer GPS, nice big screen and text to speech (TTS).  It doesn’t have Bluetooth, MP3, or any other non-GPS functionality.  I would have liked having Bluetooth, so I could use it as a speaker for my cell phone, but that would have jacked the price outside the range we wanted to pay.

It worked pretty well and made it a lot easier to get around Cape Cod.  It did have an annoying habit of trying to send me down a one way street the wrong way, but after I ignored that direction a few times it stopped sending me that way.  That would be a fault of the NAVTEQ map used by the Magellan, if you look at Pine Ave on Google Maps (also using NAVTEQ maps), it’s not labeled as one way for any stretch of it.

While writing this post, I visited the NAVTEQ site and found that they have a form for submitting feedback on their maps.  I decided to let them know about the missing one way restriction.  The first thing that I found is that the feedback page fails with Firefox 3.0.1.  As part of the data entry, it loads a map based on the address that you are reporting feedback on.  With FF3.0.1, it never loaded the map and just displayed the text “Please wait. Map applet is loading”.  After a few attempts, I filed it under the “EPIC FAIL” category and launched the page with IE7.  It worked with IE7 and I was able to report the missing road restriction without any problems.  When it prompted me for the GPS device, it did not have the 1412 model listed, but I was able to enter it in manually.  When I tried to save the report, it threw an IE runtime error message: “Object doesn’t support this property or method”, but it processed the report.  Many of the links on NAVTEQ’s site threw runtime error dialogs, it got annoying after a while.

After some poking around on the Internet, I found a site called www.gpsreview.net, a fan site for GPS devices.  They had a section for Magellan RoadMate units, so I started browsing the messages.  I learned that there was a newer firmware, 1.34, than the one I had (1.28).  According to the message thread, the newer firmware has a faster touch screen response time plus a few other features.  In this case, faster is better, so I grabbed the firmware.

The 1412 has a standard mini-USB connector, but does not come with a USB cable.  If you have any device that uses mini-USB, then the odds are that the cable that came with that device will work. I keep a USB to mini-USB cable with each PC that I use, it just makes life easier.  If you need one, look for a “A Male” to “Mini B, 5 pin Male”.  Something like this.  It shouldn’t cost more than a couple of bucks.

The installation instructions included a note about having the battery in the 1412 at full charge before upgrading.  Don’t ignore that warning.  When you run the firmware upgrade, you connect the GPS unit to your PC, but you have to unplug it after the firmware has been transferred over to the device.  When the GPS unit is plugged into a PC, it runs in a “USB Connected” mode and normal GPS functionality is disabled.  It has to be disconnected from the PC to complete the firmware upgrade.  That means it’s on battery power.  Magellan claims a full charge will last up to 3 hours.  The only way that is going to happen is if the screen brightness is turned all the way off.  You’ll want to lower the brightness down to the dimmest level that you can still read before doing the upgrade.  After you unplug the cable, the installation process continues on the device and it rebotts a couple of time.  It didn’t take long and the battery held out.

After the upgrade was complete, I started up the 1412.  I brought up the map and the there were no streets.   It appears to be lost.  Being a fan of The IT Crowd, I immediately turned it off and back on again.  No change, just a relatively blank display.  I pressed the little tow truck icon at the bottom of the display to bring up current location.  That showed a blank street location and the while the latitude was correct, the longitude was very close to 0.  Well a longitude of 0 degrees is at the prime meridian, so I started zooming out on map display.  When I zoomed out far enough, I could see that I was in the Pyrénées, in central France.

That’s not good.  France is a non-trivial distance from upstate New York.  I was doing this inside an office building during a heavy rainstorm and it looked like the GPS unit was only picking up four satelites, where it usually picks up quite a bit more.  After some poking around on the unit, I found a “Set GPS Position” option where I could enter in a street address to reset it’s position.  I entered in my office street address and the map went back to normal.  I though that I had somehow toasted the GPS device, but it was back to normal.  All in all, it’s a cool device.

Live from Daryl Hall’s House

I came across an interesting site while looking for something else on Wikipedia….
 
Daryl Hall, of Hall & Oates fame, has had a hobby of restoring historic homes here in the US and in England.  Down in Dutchess County, he restored two houses and moved them from separate locations to one property.  These houses were built in 1771 and 1780 and have been joined together by a common room that Hall now uses to perform with friends for a monthly webcast.  The site is http://www.livefromdarylshouse.com and the streaming content is free.  You can download the content for a fee as mp3 or iPod friendly video.  It’s worth watching just hear his older stuff reinterpreted by guest artists.  And it’s not just his material, I got a big kick watching him sing “Cruel to be Kind” with Nick Lowe.
 
I’m not huge Hall & Oates fan, but they had some really good songs back in the 70’a and he can still hit the notes.  I really liked Hall’s Sacred Songs solo CD from 1980 more than I like the duo’s music.  I’m not a big fan of watching musical performances on the computer, but this is good stuff.
 
The interesting thing is Hall is paying the all of the costs out of his own pocket.  He’s going straight to the consumer without involving any record company.  Of course, the more popular his site gets, the bigger his bill will be for the bandwidth costs.  The production quality is very good, that can’t be cheap.  I hope than he can make enough money from the on-line sales to keep doing new shows.  Since he’s his own publisher, he can do as many or as few webcasts that he wants.  That’s one of the cool things about this format, he has no obligation to perform on any arbitrary schedule.  That should keep the shows fresh.

Bonus TVUG presentation by John Papa

On Tuesday, July 22nd, we will be having a bonus Tech Valley User Group meeting.  We usually skip the summer months, but John Papa is going to be in town and he offered to do a presentation how to consume WCF web services from Silverlight applications

John is currently working on new book, Data Services with Silverlight 2, so we get to see first hand what will be in the book.  He posted on his blog what the presentation will cover, anyone interested in doing Silverlight development should attend. 

John has posted an abstract for his presentation:

Silverlight enables developers to use their .NET and XAML skills to develop Rich Internet Applications and to build data driven Silverlight applications that communicate with multi-tier architectures. This session will show how to build Silverlight 2 applications that communicate with and consume REST services and use LINQ to XML to manage XML content, show the various data binding techniques, and show how to use WCF to talk to various middle tier services including custom entity models and the Entity Framework.

If you would like to attend, TVUG meets at VersaTrans Solutions, at 4 British American Blvd, in sunny Latham.  Click here for a map of how to get there.

Dr. Horrible’s Sing-Along Blog is coming to an Internet near you.

I can’t pass this one up.  “Dr Horrible’s Sing-Along Blog” is a direct to the Internet miniseries from the mind of Joss Whedon.  It stars Neil Patrick Harris and Nathan Fillion and NPH has a knack for playing crazy.

Act One will start streaming on July 15th.  Act’s Two and Three will follow on July 17th and July 19th, respectively.  After the 19th, they will be take down, but they will be available for purchase at a later date/

Here’s a teaser:

Teaser from Dr. Horrible’s Sing-Along Blog on Vimeo.

Thanks to Marc for the link.

Handling collation sequences with temporary tables and table variables with SQL Server

When building complex stored procedures that span multiple tables, you will probably need to store some intermediate results in a local buffer and process them before returning the final output.  SQL Server lets you do this through temporary table and table variables.  

Usually table variables offer faster performance (less locking and logging are required), but they have more restrictions than temporary tables.  Common to both types is where they are located.  When you create a table variable or temporary table, it gets created in the tempdb database, not in the current database.  This can affect the collation sequences applied to character fields  If the SQL Server was installed using one collation and your database uses a different collation, joins from tables in your database with temporary tables will fail if you join on character fields.  The tempdb database will use the server default collation sequence. That collation may not be the same collation used by your database if your created your database on a different server that used a difference collation.  The server collation is used for all of the system databases (including tempdb) and for any newly created user databases.  Databases that are attached or restored from a backup keep the collation that they were created with.

Gregory Larsen posted some sample SQL code on sqlservercentral.com that will demonstrate the error.  If you run the following SQL:

create table #a (char_set1 varchar(50) collate Latin1_General_CI_AS) 
create table #b(char_set2 varchar(50) collate Latin1_General_BIN)
insert into #a values ('collate')
insert into #b values ('collate')
select * from #a join #b on char_set1 = char_set2

You will an error message like the following:

Msg 468, Level 16, State 9, Line 5

Cannot resolve the collation conflict between “Latin1_General_BIN” and “Latin1_General_CI_AS” in the equal to operation.

That example is used to show the type of error you would get in your code.  The actual code that would throw that error would be comparisons between character fields in a temporary table/table variable and with a permanent table in your database.  If you are deploying databases to servers where the server’s default collation sequence could be different than the collation sequence used by your database, then you want to add “COLLATE database_default” to all of your character field definitions when you define a temporary table or table variable.  Using “COLLATE Database_Default” will assign the collation sequence of the current database to the field.  This will allow field comparisons between character fields in temporary tables/table variables and permanent tables to execute with triggering the “collation conflict” error.

Instead of using syntax like:

create table #a(SomeID integer, SomeCharValue varchar(20)) 

declare @a TABLE(SomeID integer, SomeCharValue varchar(20))

Use the following:

create table #a(SomeID integer, SomeCharValue varchar(20) COLLATE database_default) 

declare @a TABLE(SomeID integer, SomeCharValue varchar(20) COLLATE database_default)

This will work no matter what the collation sequence for either tempdb or your database.  You don’t need to query the server to check what the sequence.  The big limitation is that you have to explicitly define the columns in the temporary table.  If you were using SELECT INTO syntax to create the temporary table implicitly by the columns in the SELECT statement, the temporary table will use the collation of tempdb.  You would need to structure the SQL.  For example, if you were using syntax like this:

select name, crdate, filename
into #a
from master.dbo.sysdatabases
drop table #a

You would need to rewrite it to look like this:

create table #a(
name sysname collate database_default,
crdate datetime,
filename nvarchar(260) collate database_default
)
insert #a(name, crdate, filename)
select name, crdate, filename
from master.dbo.sysdatabases
drop table #a

You pay a small penalty by the extra SQL to explicitly define the temporary table, but you gain with having code that will work on any server.

A nice article about using Custom Attributes in your own code

I was poking around the blogs posts on the Falafel site and I came across a nice concise example by John Waters of how to define and use custom attributes in your own code.  His example shows how to use custom attributes as a simple way to define tag values in sub classes without having to write the same code over and over again for each subclass.  It’s a short enough read that there’s no point in excerpting any of it here.

Still time to enter the Darwin Race of Languages

Do you think that hand tweaked C code runs circles around managed code (C#, Java, VB.NET)?  Or do you think that your Just In Time compiled managed code gets the job done more securely than those bit sniffing dinosaurs?  Anxious to prove that Delphi is still relevant?  Want to win some cutting edge component libraries from DevExpress?

If you answered “yes” to any of those questions, there is still time to enter the Darwin Race of Languages.  The brainchild of Sevensteps CTO Bart Roozendaal, the Darwin Race is pitting a slate of challengers using the compiler and IDE of their choice to write a utility that will solve one of three problems.  The projects are the following:

  • Temperatures and CPU load
    A utility that will measure CPU temperatures and CPU load on remote computers
  • Multi clipboard utility
    The utility should hold several entries that were place on the clipboard earlier. In effect it should replace the clipboard.
  • GUI to a command line program
    A GUI front end for the gbak database command line driven program for the Firebird database.

Each entry will be judged on the software itself and how you documented the design and development of the application.  The goal is not to see which development is best tool, but to see what tools are the best for different types of jobs.  To see where the strengths are and where are the weaknesses of each tool, using real world types of the problems.  The documentation part will share the development process and show how different people come up with solutions for similar problems.

There are already a bunch of sponsors lined up.  DevExpress will be giving away a a license for DXperience Enterprise or the VCL Subscription.  If you wint, you get to pick one of them.  I have DXperience and most of the VCL components, you can’t go wrong with either one of them.

The race is on already, but there is still time to join.  There are a bunch of participants already signed up, covering C#, Delphi, VB, and something on the Mac platform.  Java is not yet represented, does someone out there want to pick up the Java gauntlet?