Code Snippets Database Version Support - Update

Back in September in gave notice that I was going to cease supporting CodeSnip versions up to and inlcuding v2 when adding new snippets to my Code Snippets Database (see my post "Code Snippets Database Version Support").

This post is just to confirm that the last updates to the database that will be visible in CodeSnip versions 1 and 2 have just been released today.

This also means that the online version of the database, which has similar characteristics to CodeSnip 2, will no longer be able to "see" new database updates.

So, if you want to keep on getting or viewing updates to the database you need to do one of:

  1. Update to the latest version of the CodeSnip program.
  2. View the snippets using the new Code Snippets Database – Take 2 web app, which is in early beta at present.

If it's new year's eve where you live, have a happy new year!


CodeSnip source now moving to GoogleCode

Well, I've migrated the CodeSnip project to SourceForge's new Allura forge, and I think the Subversion repo software stinks!

Lots of external links into the repo that I rely on in documentation can't be made to work as I want and the whole repo support seems badly thought out to me.

So, I'm planning to move the repo to GoogleCode, which at least I know quite well because I host some other projects there.

I'm afraid this means that the migration will take even longer than I suspected - sorry for that if you're waiting to use the repo. I'll post again as promised once the new links are active.

For the time being, downloads and bugs / feature requests will still be kept on SourceForge.

UPDATE (3rd Jan 2013): All the changes to CodeSnip's hosting are now complete - more info.


CodeSnip project moving home on SourceForge

Over the next few days SourceForge will be migrating the CodeSnip project to their new "Allura" forge.

If you access CodeSnip's Subversion source code repository or download CodeSnip updates from SourceForge you will find that the project may not be available at times.

UPDATE 1: CodeSnip's source is now going to be moved to GoogleCode. See "CodeSnip source moving to GoogleCode" for more info.

UPDATE 2: All the changes to CodeSnip's hosting are now complete - details.

Once the migration is complete the URLs used to access the Subversion repo will change. Once the new URLs are working I will post them here and on CodeSnip's download page on my website.

Sorry about the interuption but SourceForge have given notice that all projects now need to be migrated. I just hope the glitches I noted in my mini review a few posts ago have been fixed.


Checking File Preambles and Watermarks

I'm constantly having to check the preambles of files for some kind of signature (or "watermark") to check or validate the file's format.

I've re-invented the wheel many times over while doing this. Time, I thought, for a little helper routine or two.

First, a little routine to check the next N bytes from a stream for a given sequence of bytes (the "watermark").

function StreamHasWatermark(const Stm: TStream;
  const Watermark: array of Byte): Boolean;
  StmPos: Int64;
  Buf: array of Byte;
  I: Integer;
  Assert(Length(Watermark) > 0, 'No "watermark" specified');
  Result := False;
  StmPos := Stm.Position;
    if Stm.Size - StmPos < Length(Watermark) then
    SetLength(Buf, Length(Watermark));
    Stm.ReadBuffer(Pointer(Buf)^, Length(Buf));
    for I := Low(Buf) to High(Buf) do
      if Buf[I] <> Watermark[I] then
    Result := True;
    Stm.Position := StmPos;

Pass it a stream and an array containing the required "watermark" and the routine checks to see if the watermark exists at the current position† in the stream. The original stream position is restored after checking. This means that the routine can be called more than once to test for different watermarks without having worry about keeping track of the stream position.

† I designed the routine to check from the current stream position rather than the beginning of the stream because some of the files / streams my have have the required sequence of bytes offset from the start.

That's the core functionality taken care of, so how can we use the routine?

How about this generalised routine to check a file watermark or preamble?

function FileHasWatermark(const FileName: string;
  const Watermark: array of Byte; const Offset: Integer = 0): Boolean;
  FS: TFileStream;
  FS := TFileStream.Create(FileName, fmOpenRead or fmShareDenyNone);
    FS.Position := Offset;
    Result := StreamHasWatermark(FS, Watermark);

This routine is pretty self explanatory: it looks for the given sequence of bytes (Watermark) in the named file. It also has an optional parameter that lets you specify the offset of the watermark in the file.

Quite often watermarks are specified as ASCII text, so I've created an overload function to take an ASCII (actually ANSI) watermark instead of an array of bytes. Here it is:

function FileHasWatermark(const FileName: string;
  const Watermark: AnsiString; const Offset: Integer = 0): Boolean;
  Bytes: array of Byte;
  I: Integer;
  SetLength(Bytes, Length(Watermark));
  for I := 1 to Length(Watermark) do
    Bytes[I - 1] := Ord(Watermark[I]);
  Result := FileHasWatermark(FileName, Bytes, Offset);

Finally a few examples, all of which assume the name of the required file is in a string variable named FileName:

  • A zip file created by PKZip has preamble 50 4B 03 $04 in hex. So a test for such a zip file could be:
    if FileHasWatermark(FileName, [$50, $4B, $03, $04]) then
      ShowMessage('PKZip file');
  • Some versions of the old style Windows help file have the byte sequence 00 00 FF FF FF FF at offset 6. The test is:
    if FileHasWatermark(FileName, [$00, $00, $FF, $FF, $FF, $FF], 6) then
      ShowMessage('WinHelp file');
  • I'm tinkering about with the Game of Life at the moment and have found there are two versions of the Life file format - 1.05 and 1.06 - which both use the .lif file extension but have different ASCII preambles. Here's a way to distinguish them using the ASCII overload of our function:
    if FileHasWatermark(FileName, '#Life 1.05') then
      ShowMessage('Life v1.05 file format')
    else if FileHasWatermark(FileName, '#Life 1.06') then
      ShowMessage('Life v1.06 file format')
      ShowMessage('Invalid Life file format');

Hope that's useful to someone.

EDIT: Versions of these routines are now available from the Code Snippets Database.

New look blog

Just finished giving the blog a new look and feel. I hope it's clearer and less fussy than the previous version.

The syntax highlighter styling has been switched to emulate the Delphi "Classic" theme, e.g.:

procedure Foo;
  // example of highlighter
  ShowMessage('The answer is ' + IntToStr(42));

So there we are then. Comments appreciated.


CodeSnip 4 Emerges

After a lot of development CodeSnip v4 has finally been released. You can get it from the CodeSnip Download Page or from SourceForge.

Here's a list of the key new features and changes:

Snippet Handling

  • New "unit" and "class" snippet kinds that can include complete units and classes (and advanced records) in the database. Both can be test compiled and classes / advanced records can also be included in generated units.
  • Snippets from both the main and user databases can now be duplicated. This is very useful if you have created a snippet and want to create another one that shares a lot of the source code, dependencies etc.
  • The full range of Unicode characters can now be used for snippet names, descriptions etc., and for source code.
  • There is now finer control over the control of warnings in generated code via the $WARN directive.
  • The names of referenced units may now contain dots so that Delphi namespaces can be used.

User Interface

  • The new multi-tab display can display details of more than one snippet, category etc. in the main display.
  • The main menu has been re-organised and there are new menu and tool-bar glyphs.
  • The structure of snippet pages in the details pane is now customisable: various page elements can be omitted and the order of elements can be changed. Each snippet type has its own page customisation.
  • The number of compilers that appear in the compile results table in the details pane can now be limited.
  • The colours used for snippet names and headings can now be customised.
  • Snippets can now have an optional "display name" that, unlike the snippet's name, does not need to be unique. This is useful for giving meaningful names to snippets such as overloaded functions. For example snippets ResizeRect_A and ResizeRect_B now have display names ResizeRect (TSize overload) and ResizeRect (Longint overload) which are much more meaningful.
  • Snippet descriptions can now be formatted and can contain multiple paragraphs, just like "extra" text. Both are edited using the new Markup Editor.
  • Syntax highlighting of source code of user defined snippets can now be switched off on a per-snippet basis. This mainly of use for "freeform" snippets that may be used for notes or snippets in languages other than Pascal.
  • Information about how a snippet from the online Code Snippets Database was tested is now displayed by means of a glyph at the top right of the detail pane.
  • The Welcome page has been completely redesigned to be cleaner and to provide more useful information.

Test Compilation

  • You can specify the paths to be searched by compilers when looking for used units. This lets you compile snippets that use units other than those provided in the Delphi VCL and RTL. For example you could specify the path to the Indy components if you need to reference them from your snippets.
  • Results of test compilations now appear in a dialogue box instead of in a tab in the detail pane.

Other Features

  • There is a new option on the Tools menu that checks availability of new versions of CodeSnip.
  • Text and compiler searches can now optionally refine a previous search rather than always searching the entire database.
  • The contents of a category can now be printed.

For other features of v4 please read the change log for release 4.0.0 and all preceding pre-releases, including alpha (preview) and beta releases and release candidates.


Aaargh - Another BAD CodeSnip Bug - Now Squashed

You couldn't make this up.

No sooner had I released CodeSnip 4 release candidate 2 to correct a nasty bug relating to snippet display names than I read a comment in the blog about another new, and even worse, bug.

This one affects clean installs of CodeSnip, i.e. installs where there was no previous version of the program installed.

It would either crash on start-up or to look into a permanent loop using a lot of processor time. Bad because it means no new users could install it. This bug goes back to beta 2.

I've rushed out a patch for this problem - CodeSnip 4 release candidate 3. You can get this on SourceForge or download it direct from my website.

If you are using any of the betas or release candidates you really SHOULD update now.

I find it strange that no-one has reported this until now, and that I didn't find this bug while testing the installer. But at least it got reported before the final version gets released - big thanks to the anon reporter. I guess people using the betas tended to be existing users so the bug didn't show up.

I'm deeply embarrassed by this :-(


CodeSnip v4.0 RC-1: Nasty Duplicate Display Name Bug + Fix

Well, this is embarrassing.

Only a few days after CodeSnip 4 RC 1 became available I've discovered a nasty bug that causes the user interface to crash if more than one snippet has the same display name. The bug also applies to Beta 1 and Beta 2.

It's really nasty for two reasons:

  1. Its easy to trigger the bug. All you have to do is to accept an edit where you have created a display name that duplicates an existing one. Once you've pressed OK its too late. Worse, if you duplicate a snippet CodeSnip automatically duplicates the display name if the snippet has one. Once again pressing OK is all that's needed.
  2. Because the bug is reported as a fatal error, CodeSnip helpfully tries to save the database before terminating. In the case of this bug it usually succeeds, which means the duplicate display name is saved in the database. When CodeSnip is restarted it instantly crashes again because its UI can't display the duplicate display names.

I've rushed out a fix that you can get from SourceForge and from my website. Please download it now if you're using one of the affected editions.

Even if you've already been bitten by the bug all is not lost. Just download and install the fixed version (or any later release) and simply run it. It will be able to load the database, with the duplicate display names, and you hopefully won't have lost any data.

You'd be well advised to backup the database before carrying on (Database > Backup User Database menu option). But you do that anyway don't you!!

Really sorry about this one folks!

The linked download also incorporates another major bug fix.


SourceForge Allura - First Impressions

Now that SourceForge are promoting their new Allura forge and are encouraging projects to move to it, I thought it was time to dip my toe in the water.

I have a tiny project called CompFileDate (a file date comparison utility) that I keep on SourceForge mainly as a test-bed. I thought I'd migrate that to try out the new platform before taking the plunge and migrating the much larger CodeSnip project.

Here are my first impressions.

The new interface is very clean and looks promising. But it is quite inconsistent and the menus change between a simple clear menu bar on some pages to a kind of toolbar on other pages. Here's part of the menu as displayed when viewing the summary page.

And now here's what it looks like when the Wiki item is clicked:

Not only has the style changed but, in the first menu you get to the download files page from the "Files" item. In the second menu you get there from "Downloads".

It confused the hell out of me at the start, and still catches me out now.

The repo browser, at first sight, it hugely improved. This:

instead of this:

And navigating is easier and more intuitive. Now, if you click on a file you see the file (yeh!), whereas before clicking on a file gave you a list of revisions and you had to click a view link to see the file itself. Conversely, if you clicked a revision number next to a file you saw the file! Love to know who thought of that one. So the new interface is a huge improvement in that respect.

The wiki is a massive improvement on the old version. It seems much quicker and uses the simple Markdown for markup. I might actually start using it now.

I haven't got issue tracking or news activated for CompFileDate, so I can't say anything about that.

There are a couple of problems with speed:

  1. The system seems slow to update: as I write on 16th September, the summary page is showing the last update as the 14th where in fact I updated only about an hour ago. And it keeps getting revision numbers out of sync.
  2. The whole system is s-l-o-w. But then again I've always found the main problem with SourceForge is its speed: I just hoped it would improve.

The file manager is pretty much unchanged from the old interface, and all the better for that. Nice.

One of the other issues that may be more or less of a problem for you is that the URLs used to checkout from the Subversion repo have changed. That's not too much of a problem for CompFileDate but it will mean quite a few documentation tweaks for CodeSnip.

I'm really hoping many of my gripes are down to teething problems, because in general I like Allura. Or rather, I think I'm going to like it.

Will I be migrating CodeSnip? Think I'll wait a while.

By the way, if you use CompFileDate, the new checkout URLs are given on the project's SourceForge Code page.

Online Code Snippets Database Reverts to Childhood

For a time now the web based Code Snippets Database has been lagging behind the CodeSnip program in the kinds of snippets it can understand and include in compilable units.

The compromise position was that it would display any snippet, but would not include some of them in compiled units. These included:

  • Overloaded functions and procedures
  • Simple types
  • Constants

CodeSnip 4 is due to make matters worse by introducing class / advanced record types and units, neither of which this venerable web app can handle.

It's a lot of work to bring the database up to date, so I've decided to replace it with an all-new Web 2.0 app. In the meantime, I've decided to restrict the old app to displaying only those snippets it understands. In effect, it is now compatible with the CodeSnip 2 way of handling the online database.

Sorry if this is inconvenient. If you need the full database, please download the latest version of CodeSnip, which can display everything and can also create units for all the snippet types.

One side effect of this is news items about new snippets that have been added to the database can't always provide a link that displays all the snippets if they include any that can't be displayed by the on-line database.
This is no longer the case: any snippet referenced in a news item is now displayed in a custom web page.


Code Snippets Database Version Support

When updating the Code Snippets Database, I have always maintained compatibility with all versions of my CodeSnip program: as many snippets as possible have been available in all versions of CodeSnip.

As CodeSnip has developed, and more and more features have been added, supporting them all has become something of a burden.

So I've decided to stop releasing snippets that support CodeSnip v2 and earlier from the end of the year.

This means that, as of 1st January 2013, any new snippets added to the database will not be visible in CodeSnip v2 and earlier. Existing snippets will continue to be available - I'm not planning to take anything down that's already there.

If you're one of the few remaining users of the early CodeSnip versions it's really time to update.

There are differences between snippets used in CodeSnip v3 and the new v4, but these are much easier to manage, and so all snippets will continue to be available for both versions, unless they need the new features introduced in v4, such as units, class types etc.


A Fix for TWebBrowser's Irritating Click Sound

For a a long time now, my CodeSnip program's propensity to make little clicking noises when some pages are displayed or some internal links are clicked has been irritating me to death.

Of course, this is because CodeSnip uses Delphi's TWebBrowser control to display the main part of its UI: and TWebBrowser wraps the Internet Explorer display engine, and that's where the well known clicks come from.

Many people say that the click is part of the expected user experience, but while I agree for apps that present a noticeably web inspired interface, I don't think it's helpful in apps like CodeSnip that simply use the browser control as a way of diplaying a richly formatted user interface: users aren't supposed to even know there's a browser control there.

Any road up, I've finally found how to turn off the dreaded click using a proper API rather than a kludge that hacks about the registry. I've seen this solution discussed for VB and C#, but not for Delphi, so here's the solution I'm using.

It uses the CoInternetSetFeatureEnabled function exported by UrlMon.dll from Windows XP SP2 / Windows Server 2003 and Internet Explorer v6.0.

My app needs to also support Win 2K, so I can't import the function statically - we need a dynamic approach. This is what I came up with:

First declare a type for the imported function and a global variable to reference the function:

  TCoInternetSetFeatureEnabled = function(
    FeatureEntry: DWORD;
    Flags: DWORD;
    Enable: BOOL
  ): HResult; stdcall;
  CoInternetSetFeatureEnabled: TCoInternetSetFeatureEnabled;

Now declare a function to use in case we can't import CoInternetSetFeatureEnabled from UrlMon.dll:

function CoInternetSetFeatureEnabledFallback(FeatureEntry: DWORD;
  Flags: DWORD; Enable: BOOL): HResult;  stdcall;
  Result := E_NOTIMPL;

This function does nothing except return E_NOTIMPL to show that it is not implemented.

Next comes a procedure that is to be called from the unit's initialization section to load CoInternetSetFeatureEnabled if possible and if not to use CoInternetSetFeatureEnabledFallback:

procedure Init;
  LibHandle := SafeLoadLibrary('urlmon.dll');
  CoInternetSetFeatureEnabled := nil;
  if LibHandle <> 0 then
    CoInternetSetFeatureEnabled := GetProcAddress(
      LibHandle, 'CoInternetSetFeatureEnabled'
  if not Assigned(CoInternetSetFeatureEnabled) then
    CoInternetSetFeatureEnabled := CoInternetSetFeatureEnabledFallback;

At the end of the unit we need the following initialisation and finalisation code:


Now we have an implementation of CoInternetSetFeatureEnabled that we can call safely, we can switch off those horrid little clicks by calling the routine with the necessary parameters:

procedure SwitchOffBrowserSounds;
  SET_FEATURE_ON_PROCESS                  = $00000002;

The FEATURE_DISABLE_NAVIGATION_SOUNDS is the important value here: it switches off the IE ticks and is one of a number of "feature controls" that can be passed to CoInternetSetFeatureEnabled. You can see the full list at http://msdn.microsoft.com/en-us/library/ms537169.

SET_FEATURE_ON_PROCESS makes the change only for the current process so we don't interfere with the settings that apply in other applications. Again, there are flags for different options: see http://msdn.microsoft.com/en-us/library/ms537168.

Of course this code has no effect if UrlMon.dll does not export CoInternetSetFeatureEnabled, but at least it fails gracefully.

At last peace reigns over the CodeSnip UI! (Or rather, it will when v4.0 beta 3 is available).


More help with CodeSnip and Delphi XE2 please

A user has reported that CodeSnip fails to compile snippets with Delphi XE2 and issues an error that suggests it can't find the required .dcu files.

EDIT:What follows was not the problem at all! It was in fact the VCL etc. units that couldn't be found and is to do with XE2 namespaces. The problem, and a work around, has been written up in this FAQ. (Actually this should a a TAQ - twice asked question!)

This may be because XE2 uses different folders for .dcu files depending on whether it is compiling for 32 bit or 64 bit.

Because I don't have XE2 I'm looking for someone who does to check if snippets compile OK on their system (requires CodeSnip v3.9.0 or later).

Additionally, I'd like you to also check if setting the -$N command line switch enables compilation to work, whether or not you find snippets compile OK.

To do this from CodeSnip, use the Tools | Configure Compilers menu option to display the Configure Compilers dialogue box. Select Delphi XE2 in the left hand list then select the Command Line tab. In the Add or edit switch edit box enter -N0.\ (that's dash N zero dot backslash) and press the Add button, then OK to accept changes.

You can revert the above change by selecting the -$N0 switch in the Switches list and pressing the Delete button.

Most grateful to anyone who can help with this. Please use comments or contact me via my website with any answers.


Last call for CodeSnip 4 feature requests

The last of the CodeSnip 4 Preview releases is out and the next planned release will be the first beta, sometime around August I guess. That should be feature complete with a bit of luck.

That means I'm closing requests for new features at the end of this month - i.e. 30th June. So if you really want to see a new feature in CodeSnip v4, then now's the time to ask. Just head over the Feature Request Tracker on SourceForge.

Not guaranteeing anything, but your idea may get accepted for v4.


MPL v2 for me

The Mozilla Public License v2.0 was officially released a few weeks ago. I never though I'd get excited about a source code license, but I have (well nearly)!

I have always been attracted to the less restrictive nature of the MPL compared with the Free Software Foundation licenses (GPL / LGPL).

Although I personally find the GPL too restrictive - I don't want to insist that my code can only be used in other GPLd project - I do want to ensure my code never goes closed, and that it is always freely available. Therefore some of the most permissive license are out of it for me.

So I have settled on the MPL v1.1 for most of my code. The big downside though is that MPL 1.1 code isn't compatible with the GPL, so my code can't be used in GPL projects. I can get round this by changing to the MPL / GPL / LGPL tri-license, but that has its own problems.

So, I'm pleased that the MPL v2 is now GPL compatible, is lot simpler to use than tri-licensing and doesn't have the same problems. Over time much of my code will be changed over to MPL v2, and all new code will use it.

Until that day, if it ever comes, if you want to use some of my code in a GPL / LGPL project, let me know and I'll change it over to MPL 2 for you.

PasHi overhaul

I've started working on a major overhaul of the PasHi Pascal Syntax highlighter adding long-overdue parsing updates and Unicode support, along with a bunch of new commands.

If you use it and would like to see a new feature, please create a new issue in the PasHi issue tracker and change the "Type-Defect" label to "Type-Enhancement". Just delete the template text and describe the feature you want.

The source code for the overhaul is in the GoogleCode Subversion Repository in the branches/v2-dev tree.


Changes to CodeSnip SVN Repo

I've made some changes to the CodeSnip Subversion source code repository on SourceForge that you need to know about if you access the repo:

  • The v4 development code has been integrated into the trunk. All development effort is now on v4, so it makes sense to move it here.
  • The v4-dev branch is now redundant and has been removed.
  • v3 maintenance code has been moved to a new branch named 3.x. All bug fixes to v3 will happen here. v3 is now in maintenance only, and no new features will be added, unless someone is very persuasive.