Separate strings by words or tokens  

Send By: Radikal (Q3 Team)
Web : http://www.q3.nu
Email: radikal@q3.nu
Date: 08/09/20

Tip accessed 918 times

 


Here you have two useful functions to separate a string containing several words separated by one character (or one string).
It can be good us to care in our database a file separated by comas, for example.
GetTokenCount gives us the token number (words) that contains the string
GetToken returns us the token that we specify him.

An example that separates a string with several words separated by comas and it goes them adding in a TMemo:

  • Put a TMemo (Memo1 and a TButton (Button1) in your form
  • Put this code in the OnClick of Button1:


     procedure TForm1.Button1Click(Sender: TObject);
    
       function GetToken(Cadena:string; Separador:char; Token:integer):string; overload
       {
       Cadena     es la string en la que buscar el token
       Separador  es la string que separa cada token
       Token      es el número de token que buscamos
    
       Cadena     is the string in that look for the token
       Separador  is the token separator
       Token      is the desired token number
       }
       var
         Posicion:integer;
       begin
          while Token > 1 do begin
            Delete(Cadena,1,Pos(Separador,Cadena));
            Dec(Token);
          end;
          Posicion:=Pos(Separador,Cadena);
          if Posicion=0
          then result:=cadena
          else Result:=Copy(Cadena,1,Posicion-Length(Separador));
       end;
    
       function GetToken(Cadena, Separador: String; Token: Integer): String; overload
       var
         Posicion: Integer;
       begin
          while Token > 1 do begin
            Delete(Cadena, 1, Pos(Separador,Cadena)+Length(Separador)-1);
            Dec(Token);
          end;
          Posicion:= Pos(Separador, Cadena);
          if Posicion = 0
          then  Result:= Cadena
          else  Result:= Copy(Cadena, 1, Posicion-1);
       end;
    
       function GetTokenCount(Cadena:string; Separador:char):integer; overload;
       var
          Posicion:integer;
       begin
         Posicion:=Pos(Separador,Cadena);
         Result:=1;
    
         if Cadena <> '' then begin
           if Posicion <> 0 then begin
             while Posicion <> 0 do
             begin
               Delete(Cadena,1,Posicion);
               Posicion:=Pos(Separador,Cadena);
               Inc (Result);
             end;
           end;
         end else Result:=0;
       end;
    
       function GetTokenCount(Cadena, Separador: String): Integer; overload;
       var
          Posicion: Integer;
       begin
         if Cadena <> '' then begin
           Posicion:= Pos(Separador, Cadena);
           Result:= 1;
           while Posicion <> 0 do begin
             Inc(Result);
             Delete(Cadena, 1, Posicion+Length(Separador)-1);
             Posicion:= Pos(Separador, Cadena);
           end;
         end else
           Result:=0;
       end;
    
    
     var
        SeparadaPorComas :string;
        n                :integer;
     begin
       Memo1.Lines.Add('Ejemplo con separador de 1 caracter');
       SeparadaPorComas:='uno,dos,,cuatro,cinco';
    
       for n:=1 to GetTokenCount(SeparadaPorComas,',') do
         Memo1.Lines.Append( GetToken(SeparadaPorComas,',',n) );
    
       Memo1.Lines.Add('Ejemplo con separador de varios caracteres');
       SeparadaPorComas:='uno[SEP]dos[SEP][SEP]cuatro[SEP]cinco';
    
       for n:=1 to GetTokenCountS(SeparadaPorComas,'[SEP]') do
         Memo1.Lines.Append( GetToken(SeparadaPorComas,'[SEP]',n) );
     end;
    




    The two functions are defined with the overload directive, for optimization in function of the length of the separator used.

    If we call the function with:

     UnToken:=GetToken(Cadena,',',2);
    



    We will use the version for char separator
    But, if we call with:

     UnToken:=GetToken(Cadena,'###',2);
    



    We will use the string separator version of the function

    This tip contains ideas and code of these users:
    Ismael S.R. (ismaeltc AT maptel DOT es)
    Carlos G
    Savierhs E. Lopez Arteaga

    Thanks


    Updated at 08/09/2005