Back to discussion

POP object -> attachment -> ASP

Last post was 29 Jan 2003 at 06:15
vb6 Zaanstad, Netherlands
  • 18 years ago

    Hi all,


    I have a problem wit a POP email class object that I created in VB.


    I am able to read the attachment of a message into a string.
    When I save that string to a file then it's a valid file (same as the original)
    So that part is OK.


    Now I'm working on a webmail client that is using this POP object.
    The problem is that the strings are in memory as 2 bytes per character.
    When I dump that to the browser then it thinks that it's receiving a file that is 2 times as big as the original with every other byte a chr(0).


    I was able to create some sort of solution by creating a loop and use the ChrB of the Ascii value but that one realy takes long if the attachment is big.


    I tried to move the code to the POP class object but was not successfull. what I tried is:
    1. Let it return in a string again but then two bytes in a char.  I could not assign 2 bytes to one character.
    2. I tried a byte array but then you still need a loop in the asp page for displaying the attachment.


    Does anyone know a better solution for passing on that file to the browser?


    I don't want to save the file to disk and then redirect. That one also works and is faster but it's hard to clean it up.


    Here is the working ASP code that I used for displaying the attachment

    Code:

    <%
      ' The POPmail object is already created in the session using a global.asa
      NR = request("MessageNR")
      ANR = request("AttachmentNR")


      With response
         .Buffer = True
         .Expires = 0
         .Clear
         .ContentType = POPmail.AttachmentContentType(NR, ANR)
         .AddHeader "Content-Type", POPmail.AttachmentContentType(NR, ANR)
         .AddHeader "Content-Disposition", "internal; filename=" & POPmail.AttachmentName(NR,ANR)
         .AddHeader "Pragma","no-cache"
         .AddHeader "Expires","0"
         .AddHeader "Cache-control","no-cache,must-revalidate,no-store"


         ' We have to convert the 16 bit characters to 8 bit characters.
         strAttachment = POPmail.Attachment(NR,ANR)
         for i = 1 to len(strAttachment)
            .BinaryWrite ChrB(Asc(Mid(strAttachment,i,1)))
         next


         .Flush
         .End
      End With


    %>



    If anyone wants to try it out then there is an alpha version available.
    You can download it from here
    This is a slim install. If you are not able to run Site Skinner then first install the full version from here


    Instructions are easy:
    Install it on your web server
    copy the directory Program files\site skinner\samples\webmail  to a directory in the webroot


    remember that it's an alpha and there is still a lot to do.


    Here is how it looks so far:


    Report abuse
  • 18 years ago

    edwin,


    i dont have an answer for your query but man.. i just had a peek at that siteskinner of yours... damm good stuff! when - and if - i buy myself some proper server space i'll get them to install that.. so by that time your webmail feature will work!


    am i seeing this right... 'hothtml4_researchbuild.png'? are you a programmer for web software (tushan?) or something? can we see what it will look like?


    finally whats with the name for siteskinner? when i first saw it on your ciggy i thought, some sort of template site developing tool like MS Publisher or something... but this is much more than that... its a mini-dotnet-system in a way...


    thanks again and sorry i couldnt answer your question...

    Report abuse
  • 18 years ago

    Hi,


    I am not a developer for WebSoftware but Thushan (lead developer) and I exchange lot's of info on almost a daily basis.
    That message in my inbox was a screenshot that he send me. He is investigating possabilities for the 4th version of HotHTML. It's just some research.


    About SiteSkinner: Maybe you'r right. When I started with it the name was ok but by now it would probably be better if I call it something like 'Internet Toolkit' or so. When I started I planned to create something that could grab data from the internet, put it into a database and then publish this data again. After that I added SMTP so that it could be published using email to distribute it. Then I adde POP because I wanted a generic way for grabbing data from an email. Then I thought. It must be easy to create a webmail client. and then here it is.


    I could not speed it up either. What I tried so far is:

    Code:

    dim bytAttachment() as byte
    bytAttachment strConv (strAttachment, vbFromUnicode)


    Code:

    Set rst = CreateObject("ADODB.Recordset")
    lngAttachmentLength = Len(strAttachment)
    rst.Fields.Append "BinData", adLongVarchar, lngAttachmentLength
    rst.Open
    rst.AddNew
    rst("BinData").AppendChunk strAttachment
    rst.Update
    strAttachmentUnicode = rst("BinData").GetChunk(lngAttachmentLength)
    rst.close
    set rst = nothing


    Code:

       strAttachment =POPmail.Attachment(NR, ANR)
       Set Stream = server.CreateObject("ADODB.Stream")
       Stream.Open
       Stream.WriteText strAttachment
       Stream.Position = 0
       Stream.Type = adTypeBinary
       .BinaryWrite Stream.Read
       Stream.Close
       Set Stream = Nothing

    Report abuse
  • 18 years ago

    hmmm 'Internet Toolkit' sounds a little lame, make it more i dunno.. cooler? WebFusion? WebSync or something along those lines?


    so both your app and his are pretty serious stuff i see? having 'research' time and 'programming' time and i'm guessing designing/planning time sometime before that? I downloaded a very early - maybe the second release? - of SiteSkinner a while ago and man... its changed so much! the new UI is much better than what you had before and its very flexible too...


    keep up the great work you two!

    Report abuse
  • 18 years ago

    What does work but does not speed things up much is this inside the pop class object:

    Code:

    Public Property Get Attachment2(lngMessageID As Variant, intAttachment As Variant) As String
    Dim strTemp As String
    Dim lngLoop As Long
      strTemp = Attachment(lngMessageID, intAttachment)
      For lngLoop = 1 To Len(strTemp)
         Attachment2 = Attachment2 & ChrB(Asc(Mid(strTemp, lngLoop, 1)))
      Next
    End Property

    This one is faster but does not do the conversion
    Code:

    Public Property Get Attachment3(lngMessageID As Variant, intAttachment As Variant) As String
    Dim strTemp As String
    Dim strTemp2 As String
    Dim lngLoop As Long
      strTemp = Attachment(lngMessageID, intAttachment)
      strTemp2 = String(Len(strTemp), Chr(0))
      For lngLoop = 1 To Len(strTemp)
         Mid(strTemp2, lngLoop, 1) = ChrB(Asc(Mid(strTemp, lngLoop, 1)))
      Next
      Attachment3 = strTemp2
    End Property

    I better try a byte array again

    Report abuse
  • 18 years ago

    After digging in to it some more I found out It's not the Unicode to binary that takes most time. It's the decode base 64


    Below there is a litle log of what I found out:


    Attempt 1 : Get the data and dump it

    Code:
      strAttachment = POPmail.Attachment(NR, ANR)
           .BinaryWrite strAttachment
    Does not work


    OK, we have to convert from a Unicode string to a binary string. Let's do it a character at a time


    Attempt 2 : convert the 16 bit character to 8 bit character

    Code:
      strAttachment =POPmail.Attachment(NR, ANR)
      for i = 1 to len(strAttachment)
         .BinaryWrite ChrB(Asc(Mid(strAttachment,i,1)))
      next
    Works but is slow (84 seconds for a 157KB file)


    Attempt 3 : Same as 2 but now the code is inside the POP object

    Code:
      .BinaryWrite POPmail.Attachment2(NR, ANR)
    Works but is even slow (102 secons for a 157KB file)
    This must be because the string concatination in the loop


    After searching for source code on Planet Source Code I found some samples that where doing a file upload.
    The conversion is then the other way around. It needed to be converted from Binary to Unicode.\
    Windows does this conversion automaticaly in some cases for instance you can write the data to
    a BLOB field in a recordset and then read it into a string. You can also write the data to a file
    and read that in again. But then the simplest is to write and then read from a file.



    Attempt 4 : Save the file to disk and then redirect to this

    Code:
      strAttachment = POPmail.Attachment(NR, ANR)
      Set fso = CreateObject("Scripting.FileSystemObject")
      Set ts = fso.CreateTextFile("D:\WWWRoot\WebMail\IE\" & POPmail.AttachmentName(NR,ANR), True)
      ts.Write strAttachment
      ts.Close
      .Redirect POPmail.AttachmentName(NR,ANR)
    Works but will leave this file on disk and for some reason i still is slow with big files (82 sec for a 157KB file)


    Attempt 5 : Same as 4 but now the code is inside the POP object.

    Code:
      POPmail.SaveAttachment NR, ANR, "D:\WWWRoot\WebMail\IE\" & POPmail.AttachmentName(NR,ANR)
      .Clear
      .Redirect POPmail.AttachmentName(NR,ANR)
    Hmmm... still slow, actually the same speed as 4. So it's the decode base 64 code that is slow


    Since writing and reading a file only improves the speed very litle we better start to improve the decode base 64 code.


    Hmm... I just saw it uses string concatanation inside a loop. Fortunately there are a couple of samples about this on Planet source code.



    Report abuse
  • 18 years ago

    That shall tach me to never use someone else its code without checking.
    Just implemented the new decode and the new decode of the 157KB file only took 3 seconds
    I also tried it with a 1.3MB file and that one only took 22 seconds.
    That was with the method with the loop in the asp

    Code:

    Private Function DecodeBase64String(str2Decode As String) As String
    ' Description:
    '   Coerce 4 base 64 encoded bytes into 3 decoded bytes by converting 4, 6 bit
    '   values (0 to 63) into 3, 8 bit values. Transform the 8 bit value into its
    '   ascii character equivalent. Stop converting at the end of the input string
    '   or when the first '=' (equal sign) is encountered.
    On Error GoTo ErrorHandler
    Dim lPtr            As Long
    Dim iValue          As Integer
    Dim iLen            As Integer
    Dim iCtr            As Integer
    Dim Bits(1 To 4)    As Byte
    Dim strDecode       As String
    Dim lPtr2           As Long


    ' Clean it up and create the destination buffer
    str2Decode = Replace(str2Decode, vbCrLf, "")
    lPtr2 = 0
    strDecode = String(Len(str2Decode) * 3 / 4 + 3, " ")
    ' for each 4 character group....
    For lPtr = 1 To Len(str2Decode) Step 4
       iLen = 4
       For iCtr = 0 To 3
           ' retrive the base 64 value, 4 at a time
           iValue = InStr(1, BASE64CHR, Mid$(str2Decode, lPtr + iCtr, 1), vbBinaryCompare)
           Select Case iValue
               ' A~Za~z0~9+/
               Case 1 To 64: Bits(iCtr + 1) = iValue - 1
               ' =
               Case 65
                   iLen = iCtr
                   Exit For
               ' not found
               Case 0
                  Exit Function
           End Select
       Next
       
       ' convert the 4, 6 bit values into 3, 8 bit values
       Bits(1) = Bits(1) * &H4 + (Bits(2) And &H30) \ &H10
       Bits(2) = (Bits(2) And &HF) * &H10 + (Bits(3) And &H3C) \ &H4
       Bits(3) = (Bits(3) And &H3) * &H40 + Bits(4)
       
       ' add the three new characters to the output string
       For iCtr = 1 To iLen - 1
           lPtr2 = lPtr2 + 1
           Mid$(strDecode, lPtr2, 1) = Chr$(Bits(iCtr))
       Next


    Next


    DecodeBase64String = Left(strDecode, lPtr2)
    Exit Function
    ErrorHandler:
         Err.Raise (vbObjectError Or 6), "POPmail", "Error in function DecodeBase64String(" & Left(str2Decode, 16) & "....) failed! The message ID does not exist."
    Resume Next
    End Function


    Report abuse
  • 18 years ago

    i also have speed problems with decoding base64


    if you find a fast solution will you share it with me?


    i'm also looking at making an email program (but not webmail).


    Report abuse
  • 18 years ago

    Hi stevesoft,


    I think the code above performs resonably OK.
    The performanse increase compared to my other was huge.
    In my Webmail i was able to save a 1.3MB file within 10 seconds
    In my old decod64 a 157KB file took 82 seconds.


    The problem was the string concatanation. That is :
      strDecoded = strDecoded & Chr$(Bits(iCtr))
    No i am creating a string of the size of the file as a buffer and use
      Mid$(strDecode, lPtr2, 1) = Chr$(Bits(iCtr))
    That improved it.


    On plannet source code you can find multiple samples about fast string functions. You will find:
    - byte arrays (which are using api to convert the pointers so that you don't have to copy the string to the array and back)
    - page buffering. An addition to what I used. The addition is that the buffer can grow.



    By the way in the base64 code I forgot to include some constans and variable declarations. Here they are:


    Code:

    ' constant & arrays used in Base64 & UUEncode encode/decode functions
    Private Const BASE64CHR As String = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/="
    Private psBase64Chr(0 To 63) As String
    Private pbBase64Byt(0 To 63) As Byte
    Private psUUEncodeChr(0 To 63) As String

    Report abuse
  • 18 years ago

    that just goes to prove that theres more than one way of doing things...


    this article may interest some of you about strings:
    http://www.aivosto.com/vbtips/stringopt.html


    its some of the funky stuff you can get with their ProjectAnalyser, its damm useful too...

    Report abuse

Post a reply

Enter your message below

Sign in or Join us (it's free).

VB 6 forum discussion

VB 6 tutorials

Quick links

Recent activity

Contribute

Why not write for us? Or you could submit an event or a user group in your area. Alternatively just tell us what you think!

Web Development

Developer Jobs

Our tools

We've got automatic conversion tools to convert C# to VB.NET, VB.NET to C#. Also you can compress javascript and compress css and generate sql connection strings.

Cancel
“The generation of random numbers is too important to be left to chance.” - Robert R. Coveyou