URL Encoding

I've being reviewing the URI encoding code from the Code Snippets Database and I realised that it doesn't comply with RFC 3986.

So here's my first attempt at some compliant code.

According to the RFC:

"the data should first be encoded as octets according to the UTF-8 character encoding [STD63]; then only those octets that do not correspond to characters in the unreserved set should be percent-encoded."

So, we define the URIEncode function to operate on the UTF8String type. It's easy to encode UnicodeString and AnsiString into UTF using the System unit's UTF8Encode overloaded functions. You can overload URIEncode to do the UTF8 conversion, but I haven't done here.

There's a nice shortcut we can take when url encoding. Remember only unreserved characters are percent-encoded. The set of unreserved characters is:

const
  cURLUnreservedChars = [
    'A'..'Z', 'a'..'z', '0'..'9', '-', '_', '.', '~'
  ];

All other characters are percent encoded. But what about any UTF-8 continuation bytes? Well, by definition these have value > $80. And all the unreserved characters have ordinal value < $80. This means that no legal continuation character can be an unreserved character.

Therefore any byte in the UTF-8 string can be treated the same regardless of whether it's a lead or continuation character: i.e. we percent encode it if it's not an unreserved character.

Here's the function:

// Assumes Defined(UNICODE)
function URIEncode(const S: UTF8String): string; overload;
var
  Ch: AnsiChar;
begin
  // Just scan the string an octet at a time looking for chars to encode
  Result := '';
  for Ch in S do
    if CharInSet(Ch,  cURLUnreservedChars) then
      Result := Result + WideChar(Ch)
    else
      Result := Result + '%' + IntToHex(Ord(Ch), 2);
end;

This, and more similar routines, are available (and may even be evolving) in my Delphi Doodlings repo. View the code (see UURIEncode.pas).

Source code highlighting by DelphiDabbler PasHi.

Comments

Post a Comment

Comments are very welcome, but please be aware that I moderate all comments, so there will be a delay before your comment appears.

Advertising spam and the rare abusive, hateful or racist comments will be blocked and reported.

Finally, should you have a query about, or a bug report for, one of my programs or libraries please use the relevant issue tracker rather than posting a comment to report it.

Thanks

Popular posts from this blog

Initialising dynamic arrays

When writing some unit tests a while ago I found myself needing to initialise some dynamic arrays with test data. It would be nice if we could do something like this: var A : array of Integer ; begin A := ( 1 , 2 , 3 , 4 ) ; // !! WRONG end ; but we can't. So I decided to write some functions to initialise dynamic arrays to the contents of another array, be it a constant, a literal or another dynamic array. The result was a set of overloaded routines, one for each data type I needed to handle, for example for Integer and string arrays I had: function CloneArray ( const A : array of Integer ) : TIntegerDynArray ; overload ; var Idx : Integer ; begin SetLength ( Result , Length ( A ) ) ; for Idx := Low ( A ) to High ( A ) do Result [ Idx - Low ( A ) ] := A [ Idx ] ; end ; function CloneArray ( const A : array of string ) : TStringDynArray ; overload ; var Idx : Integer ; begin SetLength ( Result , Len...
Read more

Deleting elements from a dynamic array

In my blog " Initialising Dynamic Arrays " I mentioned I have been writing some unit tests that needed numerous repetative array initialisation operations. Well, one of the other common operations I found myself writing was the deletion of items from dynamic arrays. So I decided to find a generic way of doing this. Assuming we know the index of the item to be deleted we need the equivalent of Delphi's built-in Delete procedure for strings. If you read the previous post you won't be surprised to learn that I decided to further extend the Generics.Collections.TArray class by adding to the TArrayEx class I introduced there. My first attempt emulates Delphi's Delete procedure by deleting the array element in-place: i.e. it updates the array passed as a parameter. Unsurprisingly the method is called DeleteInPlace . Here's how it goes: type TArrayEx = class ( TArray ) public .. . class procedure DeleteInPlace < T > ( var A : ...
Read more

New Array Utilities Library Unit released

Image
Yesterday I added a shiny new unit to the DelphiDabbler Code Library - the Array Utilities Unit . It lives in the ddablib/arrayutils  repo on GitHub. This unit, named DelphiDabbler.Lib.ArrayUtils.pas ,   provides a single "Advanced" record, TArrayUtils , that contains static methods that perform common operations on generic arrays. There are numerous overloaded versions of many of the methods. Every method operates on a given generic array that is passed as a parameter, either as an array of T or a TArray<T> . The unit requires Delphi XE or later. It has been tested with Delphi XE and Delphi 12. The code depends only on the Delphi RTL, so it should work for any platform supported by Delphi and be both VCL and FireMonkey compatible. I've only tested Windows 32 bit and 64 bit targets. There are far too many methods to describe in here, but you can find them all listed and explained in the comprehensive  online documentation . As a taster you'll find Methods that ...
Read more