Library tutorials & articles

Create a Site Search Engine in ASP.NET

By Stevan Rodrigues, published on 07 Feb 2004

Download.zip

UserSearch.vb

It contains the following properties:

`SearchCriteria`	The user choice of search is stored and retrieved from here
`SearchWords`	The search words used by the user is stored and retrieved from here
`TotalFilesSearched`	Total Files Searched is read from here
`TotalFilesFound`	Total Files Searched found is read from here

        '**********************************************************
        '
        ' SearchCriteria Property
        '
        ' Assign and retrieve SearchCriteria of the site
        '
        '**********************************************************
        Public Property SearchCriteria() As Searchs.SearchCriteria
            Get
                Return m_searchCriteria
            End Get
            Set(ByVal Value As Searchs.SearchCriteria)
                m_searchCriteria = Value
            End Set
        End Property

        '**********************************************************
        '
        ' SearchWords Property
        '
        'Assign and retrieve SearchWords of the site
        '
        '**********************************************************
        Public Property SearchWords() As String
            Get
                Return m_searchWords
            End Get
            Set(ByVal Value As String)
                m_searchWords = Value
            End Set
        End Property

        '**********************************************************
        '
        ' TotalFilesSearched Property
        '
        ' Retrieve TotalFilesSearched of the site
        '
        '**********************************************************
        Public ReadOnly Property TotalFilesSearched() As Integer
            Get
                Return m_totalFilesSearched
            End Get
        End Property

        '**********************************************************
        '
        ' TotalFilesFound Property
        '
        ' Retrieve TotalFilesFound of the site
        '
        '**********************************************************
        Public ReadOnly Property TotalFilesFound() As Integer
            Get
                Return m_totalFilesFound
            End Get
        End Property

        '**********************************************************
        '
        ' PageDataset Shared Property
        '
        ' Retrieve data of the entire site of the site
        '
        '**********************************************************
        Public ReadOnly Property PageDataset() As DataSet
            Get
                Return m_dstPages
            End Get
        End Property

Search Method

Actual processing of the search begins here. DataSet to store the search results is created here and ProcessDirectory method is called.

        '********************************************
        '
        ' Search Method
        '
        ' Search the entire site
        '
        '********************************************
        Public Function Search(ByVal targetDirectory As String) As DataSet
            'If the site is in English then use the server HTML encode method
            If Searchs.Site.EnglishLanguage = True Then
                'Replace any HTML tags with the HTML codes 
                'for the same characters (stops people entering HTML tags)
                m_searchWords = m_page.Server.HtmlEncode(m_searchWords)
                'If the site is not english just change the script tags
            Else
                'Just replace the script tag <> with HTML encoded < and >
                m_searchWords = Replace(m_searchWords, "<", "<", 1, -1, 1)
                m_searchWords = Replace(m_searchWords, ">", ">", 1, -1, 1)
            End If
            If m_dstPages Is Nothing Then
                m_dstPages = Searchs.PagesDataset.Create()
            End If
            ProcessDirectory(targetDirectory)
            Return m_dstPages
        End Function

ProcessDirectory Method

The ProcessDirectory loops through all the files and calls the ProcessFile method. Later, it also loops through the subdirectories and calls itself.

       '*********************************************
        '
        ' ProcessDirectory Method
        '
        ' Files in the directories are searched
        '
        '********************************************
        Private Sub ProcessDirectory(ByVal targetDirectory As String)
            Dim fileEntries As String()
            Dim subdirectoryEntries As String()
            Dim filePath As String
            Dim subdirectory As String

            fileEntries = Directory.GetFiles(targetDirectory)

            ' Process the list of files found in the directory

            For Each filePath In fileEntries
                m_totalFilesSearched += 1
                ProcessFile(filePath)
            Next filePath

            subdirectoryEntries = Directory.GetDirectories(targetDirectory)
            ' Recurse into subdirectories of this directory

            For Each subdirectory In subdirectoryEntries

                'Check to make sure the folder about to be searched 
                'is not a barred folder if it is then don't search
                If Not InStr(1, Searchs.Site.BarredFolders, _ 
                  Path.GetFileName(subdirectory), vbTextCompare) > 0 Then
                    'Call the search sub prcedure to search the web site
                    ProcessDirectory(subdirectory)
                End If

            Next subdirectory

        End Sub 'ProcessDirectory

ProcessFile Method

The ProcessFile calls the GetInfo which returns the Searchs.Page object which contains all the information of the particular file. Later, it checks if the matchcount is greater than 0 and calls the CheckFileInfo to clean up the information stored in the Page object. It then stores the file in the PagesDataset.

        '*******************************************************
        '
        ' ProcessFile Method
        '
        ' Real logic for processing found files would go here.
        '
        '*******************************************************
        Private Sub ProcessFile(ByVal FPath As String)
            Dim srchFile As Searchs.Page

            srchFile = GetInfo(FPath)
            If Not IsNothing(srchFile) Then

                srchFile.Search(m_searchWords, m_searchCriteria)
                If srchFile.MatchCount > 0 Then
                    m_totalFilesFound += 1
                    'Response.Write(srchFile.Contents)
                    srchFile.CheckFileInfo()
                    Searchs.PagesDataset.StoreFile(m_dstPages, srchFile)
                End If

            End If

        End Sub 'ProcessFile

GetInfo Method

The GetInfo method's main task is to get the data of the file. It calls the shared method Searchs.FileContent.GetFileInfo where much of the work is done.

        '*****************************************************************
        '
        ' GetInfo Method
        '
        ' File data is picked in this method
        '
        '*****************************************************************
        Private Function GetInfo(ByVal FPath As String) As Searchs.Page

            Dim fileInform As New FileInfo(FPath)
            Dim sr As StreamReader
            Dim srchFile As New Searchs.Page()
            Dim strBldFile As New StringBuilder()
            Dim strFileURL As String  'Holds the path to the file on the site


            'Check the file extension to make sure the file 
            'is of the extension type to be searched

            If InStr(1, Searchs.Site.FilesTypesToSearch, _ 
             fileInform.Extension, vbTextCompare) > 0 Then
                'm_page.Trace.Warn("File ext.", fileInform.Extension)
                'Check to make sure the file about to be searched 
                'is not a barred file if it is don't search the file
                If Not InStr(1, Searchs.Site.BarredFiles, _ 
                  Path.GetFileName(FPath), vbTextCompare) > 0 Then
                    'm_page.Trace.Warn("File", FPath)

                    If Not File.Exists(FPath) Then
                        'm_page.Trace.Warn("Error", _
                        'String.Format("{0} does not exist.", FPath))
                        'Add throw excetion here
                        '
                        '
                        Return Nothing
                    End If

                    Searchs.FileContent.GetFileInfo(FPath, srchFile)

                    Return srchFile

                End If
            End If
            Return Nothing
        End Function

Comments

John Loser

28 May 2008 at 04:09

hello all

i am trying hard to integrate the search engine code but have failed, my site is on local lan server with the root folder as a virtual folder , my site is in c# , i have placed the contents in the folder named searchdotnetc , but the code dosnt works, i modified the code made the class partial for the cs file and also tried placing the contents folder in the app code , but i still parsers only static pages and that to only in the searchdotnetc folder for the dynamic pages it gives the error

Line 273:			wcMicrosoft = new System.Net.WebClient();
Line 274:			objUTF8Encoding = new UTF8Encoding();
Line 275:			srchFile.Contents = objUTF8Encoding.GetString(
Line 276:				wcMicrosoft.DownloadData(String.Format("{0}/{1}", Searchs.Site.ApplicationPath, srchFile.Path)));
Line 277:		}

Source File: v:\SITENAMEwww\App_Code\components\UserSearch.cs Line: 275

Stack Trace:

[WebException: The remote server returned an error: (404) Not Found.]
   System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request) +338
   System.Net.WebClient.DownloadData(Uri address) +181
   System.Net.WebClient.DownloadData(String address) +26
   Searchs.UserSearch.GetDynamicFileContent(Page srchFile) in v:\SITENAME\www\App_Code\components\UserSearch.cs:275

please help .....

shaka

25 Jun 2007 at 19:01

Do you have a version of the Search Engine project, for ASP.NET 2.0, that works well with Master Pages?
sree2000

02 Jun 2006 at 17:53

Hello,

I have down loaded the SearchDotNet.zip file from the code project. I am using VS2005, and asp.net 2.0 the files are not getting converted successfully. Can you provide me a solution to integrate this search facility into my web application .
Deepthi P

31 Jan 2006 at 07:25

when i used ur solution to implement search engine for my website it is searching only words .
for example if hai is given it is finding some files having "hai" but if h is given it is showing " 0 found"
please suggest me one solution
Cassius Arulkumaran

18 Jan 2006 at 21:47

you can use text copy to copy images and text objects inside of SQL server
http://www.mssqlcity.com/Articles/KnowHow/Textcopy.htm
kody

03 Nov 2005 at 23:07

Did you find it ? Because I am also searching for something like that.
trobin7

04 Oct 2005 at 15:37

It seems I have this error as well. Do I need to give write access to the root directory or access to another file or something else?

As a test, I've given the "Everyone" full access to the root, but doesn't help

Thanks ..Troy
Stevan Rodrigues

13 Sep 2005 at 08:53

The error is related to the permissions. You will need to check the permissions and ensure that the application has permissions expecially System.Net.WebClient
Najeem M Illyas

11 Sep 2005 at 13:58

Hi I'm also facing a similar problem with the search program. I'm getting error like 404. Anybody tell me how to fix the problem, I've configured the web.config file correctly.
oramike

18 May 2005 at 15:10

hi, i have an SQL Server database with documents from applications (ms word, excel, etc). i now need to retrieve these documents in their format but don't know how to go about it.

can anyone lend a hand, please!
oramike

15 Apr 2005 at 15:30

Hi James, thanks for the help. I finally got it.
James Crowley

14 Apr 2005 at 23:17

In IIS, you either need to make the virtual directory an "application". To do this, right click on the virtual directory (in IIS admin), click properties, and click the "Create" button next to "application name".

If you don't have an application configured, then there's a set of configuration options in web.config that aren't supported - hence it saying "It is an error to use a section registered as allowDefinition='MachineToApplication' beyond application level.". If you remove the <authentication> tag entirely, then this should work (or possibly it will complain about another unsupported tag )
oramike

14 Apr 2005 at 19:44

When i tried to run the code, i get this error message:

Configuration Error.

Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.

Parser Error Message: It is an error to use a section registered as allowDefinition='MachineToApplication' beyond application level. This error can be caused by a virtual directory not being configured as an application in IIS.

Source Error:

Line 34: "Forms", "Passport" and "None"
Line 35: -->
Line 36: <authentication mode="Windows" />
Line 37:
Line 38:

Source File: c:\inetpub\wwwroot\SearchDotnet\web.config Line: 36.

I changed the authentication mode to "None", but i still get the same error message.

Is there anything i should do that i have not yet done?
pax

04 Feb 2005 at 01:23

i'm trying to search in an asp file but it doesn't work, i'm not getting the error 500, it's solved but i still not searching on the asp file, it appears like there's no match on that file but there's a match. what can i do?
tiglathpileser

09 Dec 2004 at 10:06

we make a search engine that we are in the process of porting to .net Get in touch with me if you are interested in doing some beta testing. Office document indexing is quite tricky with a lack of documentation available and pdf documents store information in a variety of strange ways. contact me: paddy phdcc.com
Hugmup

07 Dec 2004 at 18:39

I am not a Visual Studio developer, but I have devleped about 8 ASPX projects with a notepad called ScITE for my web site and I've compiled my DLLs with the command-line compiler.

I downloaded the source files. I put SearchSite.aspx in the wwwroot\website directory on my local IIS server, I put the DLL in wwwroot\website\bin, and I put the web.config file in wwwroot\website. Then I configured web.config according to the instructions, modified the HTML in the ASPX file so that it matches my web site, and changed the name of SiteSearch.aspx to zsearch.aspx to match my file naming scheme.

It works beautifully on my local IIS server, but it does not work on the production server. The error is Service Unavailable.

This is not useful.
Stevan Rodrigues

03 Dec 2004 at 21:31

My code searches for files that store content as text or removes text from html. It is not made to search for pdf and word documents.
jackthecat64

03 Dec 2004 at 13:44

Hi. I read your query to Stevan about dealing with .pdf files in his search engine (searchdotnet). I don't think he replied to you, right?

Did you solve this problem? I'm am having the same issue.

Thanks in advance for any help you can offer.

Beth
musa7290

28 Sep 2004 at 19:31

I would highly appreciate if someone points me some sample code for simple search engine\crawler\robot that can be used\extended to gather news headlines from news websites.

Basically I am highly interested to implement a small live new page like news.google.com
Thanks in Advance
Musa
fpga

17 Aug 2004 at 08:19

Hi,
I would integrate your solution in my web...but , something I don't understund...
you explain that we have to "2) Place the SiteSearch.aspx and web.config in the root of the application"

but , I allready have a web.config , for my application.
...
can you give me more information about it ?

thanks
Bob Reichard

16 Aug 2004 at 08:30

I have your Sitesearch code working fine on my development system and development server. However, on my live server I receive the following error. But if I log into the server through terminal and acces the site in a browser from that box it works fine. Any idea why this is happening? Permissions all seem to be fine.

The underlying connection was closed: Unable to connect to the remote server.
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.
[WebException: The underlying connection was closed: Unable to connect to the remote server.]
System.Net.HttpWebRequest.CheckFinalStatus() +673
System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) +140
System.Net.HttpWebRequest.GetResponse() +149
System.Net.WebClient.DownloadData(String address) +151
clearweld.Searchs.UserSearch.GetDynamicFileContent(Page srchFile)
clearweld.Searchs.UserSearch.GetFileInfo(String FPath, Page srchFile)
clearweld.Searchs.UserSearch.GetInfo(String FPath)
clearweld.Searchs.UserSearch.ProcessFile(String FPath)
clearweld.Searchs.UserSearch.ProcessDirectory(String targetDirectory)
clearweld.Searchs.UserSearch.Search(String targetDirectory)
clearweld.SiteSearch.SearchSite(String strSearch)
clearweld.SiteSearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +731
Ilkin E Jamalli

27 Jul 2004 at 23:17

I have installed on c:\wwwroot\search folder.
It works fine but it search only on c:\wwwroot\search\sites folder. How can I make it to search in c:\wwwroot\
Can you please suggest ?
Stevan Rodrigues

23 Jun 2004 at 22:15

Hi,
I have sent the updates but the admin here is too too busy to do changes. So my latest code can be found at
http://www.codeproject.com/aspnet/SearchDotnet.asp. I have uploaded the c# version there. The C# version is older version If you want to use VB.net code and since you have webmatrix then

1)You can just use the dll of vb.net search engine and place it in the bin of your project

2) Use the aspx pages in your c sharp project

3)Convert the code behind vb pages to c sharp

Steve
Sri Koerniwati

23 Jun 2004 at 10:06

Hi Steve,

I am beginner asp.net programmer. I am developing website using webmatrix, I use C#. I create a folder for all my pages then I created a virtual directory in IIS. I download your search engine vb code, I copy searchdotnet folder to my website folder then I run SiteSearch.aspx. The page was loading and when I tried to search words that are in my pages, I got this message: 0 Files found. I tried to put one of my pages in Sites folder under SearchDotNet Folder and tried again to search word on my pages and I still got the same message: 0 Files found.

Could you help me with this ? Where should I put my pages so when I run the SiteSearch.aspx, I can find words that are on my pages.... how to convert VB.net code to C#?

Thank you so much for helping me....
Stevan Rodrigues

21 Jun 2004 at 00:10

Hi,
1)You can just use the dll of vb.net search engine and place it in the bin of your project

2) Use the aspx pages in your c sharp project

3)Convert the code behind vb pages to c sharp

4)My latest code can be found at
http://www.codeproject.com/aspnet/SearchDotnet.asp

Steve
Emily Grettel

20 Jun 2004 at 23:18

Hi Steve!

Just wondering, if you knew how much of a problem it would be if we integrated your VB.NET search tool in our C# web application for our intranet... theres roughly 400 pages (dynamic and static) but i'm not sure whether things will cause problems because ours is written in C# and the search engine is VB.NET.

Also, we would be having it in the root (Eg. /search.aspx) so would this be a problem? or do we need to manually convert all to C# before we try and integrate it?

Thanks and congratulations on a FANTASTIC and truely excellent piece of work:-)
Emily Grettel

20 Jun 2004 at 23:18

Hi Steve,

Just to let you know I posted a msg about your script here:
http://www.developerfusion.com/forums/topic.aspx?id=21918
Alan

09 Jun 2004 at 01:43

Great code stevan....!!

Was just wondering if you had any ideas about working with PDF's and word documents

I am using the XML version to search an entire intranet. The site contains a large number of pdfs and documents that have embedded graphics.

The search functionality works perfectly. However, the xml file contains code that represents the graphic part of these documents. Therefore when a search is completed, the content field returns meaingless information:

Search on Marketing returns:

Title: Marketing
Content: PDF 1 3 6 0 obj endobj xref 6 36 0000000016 00000 n 0000001064 00000 n 0000001368 00000 n 0000001572 00000 n 0000001730 00000 n 0000002139 00000 n 000000266

Any ideas how this could be filtered out...???
rogeriogr

14 May 2004 at 10:48

Hi folks,

I am trying the SiteSearch script in my personal web page and it is working well except by the fact that the portuguese accents are not being showed on the search results.

For example, when showing the results ú appears as uacute; ç appears as ccedil.

Thanks in advance.
Stevan Rodrigues

07 May 2004 at 22:33

The Latest Code and images can be found at http://www.codeproject.com/aspnet/SearchDotnet.asp.
Stepan Kazakov

22 Apr 2004 at 10:22

I got your updated version and when i try to build it i encouter this problem. Any idea why?

Thanks
Stevan Rodrigues

08 Apr 2004 at 04:37

Hi,
I have updated my article n sent to the admin but there has been a delay.Just try this
http://www.freewebs.com/stevanrodrigues/searchdotnet.html
Regards,
Stevan
Manoj Kumar

29 Mar 2004 at 07:37

Dear Sir,

This is manoj and using your site search engine
and facing so many errors while using it.
can you give me the process from A-Z how to use it?

Thanks & Regards
manoj(mnj_secret@yahoo.com)
asplover

20 Mar 2004 at 12:44

hello,firend:
i come form china.i appreciate your site search engine very well. but i encount a problem that it can not search chinese word. please tell me how to solve this problem.
thank u very much!
asplover
3.20
Simone75

09 Mar 2004 at 03:38

Thank you Stevan,
Now all work correctly.
I use your dll and it works fine.
I've a c# web project so i can't integrate your vb.net project with mine using your guidelines.
In a later version of your project think about configuration of the root in wich begin the search; i think it is helpful.
Thank you very much.
Bye.
Stevan Rodrigues

09 Mar 2004 at 01:13

Well just go to the http://www.freewebs.com/stevanrodrigues/SearchDotnet.html where you will get the latest update. Cick on the third link ie..
Download latest version of demo project (Visual studio.net not required)
1) Place the SearchDotnet.dll in the bin folder in the root of the application eg: http:\localhost\abitaria.
2) Place the SiteSearch.aspx and web.config in the root of the application eg: http:\localhost\abitaria.

If you still want to integrate the code and not just dll in your page then
1)Since you already have a project in visual studio .net open the old project(vb.net web application) which is pointing to the root of the application eg: http:\localhost\abitaria.
2)Place the sitesearch.aspx in the directory "abitaria".
3)Place the components folder in root "abitaria" or any folder within "abitaria"
4)Build the code.

Regards,

Stevan Rodrigues
Simone75

08 Mar 2004 at 04:43

I'm sorry but i've problems yet.
1)I'm not able to open In visual studio .net a new project(vb.net web application) pointing to the root of the application eg: http:\localhost\abitaria because visual studio .net tell me that already exists a project pointing in that root.
2)I tried, also, to ONLY copy sitesearch.aspx and Components directory in root Abitaria but when i try to load the aspx page this error is shown:
Description: An error occurred during the parsing of a resource required to service this request. Please review the following specific parse error details and modify your source file appropriately.

Parser Error Message: Could not load type 'SearchDotnet.SiteSearch'.

Source Error:

Line 1: <%@ Page Language="vb" Trace="False" AutoEventWireup="false" Codebehind="SiteSearch.aspx.vb" Inherits="SearchDotnet.SiteSearch" debug="false" %>
Line 2: <HTML>
Line 3: <HEAD>

Source File: C:\Inetpub\wwwroot\Abitaria\sitesearch.aspx Line: 1

So i think i must change the code in sitesearch.aspx because it inherits from your SearchDotnet project

I like your SearchDotnet application and i want to use it in my web sites but i need more help.
Could you help me again?
Thank you.
Simone
Stevan Rodrigues

07 Mar 2004 at 00:46

Hi,

Do you have visual studio .net? Do you have the .Net framework?

1)In visual studio .net open a new project(vb.net web application) pointing to the root of the application eg: http:\\localhost\abitaria.
2)Place the sitesearch.aspx in the directory "abitaria".
3)Place the components folder in root "abitaria" or any folder within "abitaria"
4)Build the code.

There is a work around do all this without visual studio .net. I have a solution without visual studio send me an email and I will let you know.

Regards,

Stevan Rodrigues
Simone75

05 Mar 2004 at 10:46

Hello,
i want to knows how i can integrate your site search web application in my web application.
I have a virtual directory called "abitaria", you use your virtual directory called "searchdotnet".
I want sitesearch.aspx in my virtual directory and i want the search in my root "http:\\localhost\abitaria" not in "http:\\localhost\searchdotnet\site".
Can you help me???
thank you
email me
Stevan Rodrigues

29 Feb 2004 at 05:53

I have sent the latest update to admin till then you can try the code at
http://www.codeproject.com/aspnet/SearchDotnet.asp
Also you can get in touch with me by email.
Stevan Rodrigues

26 Feb 2004 at 00:36

Well just run a search for 'error' you will get the page listed there. This is because your page may require some parameter(querystring or form) or cannot be read directly. Try calling from the web using that path in address of your browser.
Dennis St. John

26 Feb 2004 at 00:20

I have been successful in running this app in a virtual directory in IIS (i.e. /SearchDotNet), however, what if I want to search the files in the root directory as well? I copied the app to the root and can't seem to get it to work as it does within a virtual directory underneath of the root. Any help or suggestions would be greatly appreciated.

Thanks!

Dennis - PC1, Inc.
Joshua Stengel

25 Feb 2004 at 13:25

I made a modification as show in another post
and that got rid of the error but it still doesn't
search the dynamic content. I am investigating...

See post for 500 and 401 errors:
http://www.developerfusion.com/forums/topic-19327
Joshua Stengel

25 Feb 2004 at 12:16

When searching dynamic .aspx pages I get a 404 error.
Any thoughts?

I have the following lines in my web.config file and the
engine runs fine in directories that do not contain
dynamic content.

web.config lines:

<add key="FilesTypesToSearch" value=".aspx" />
<add key="DynamicFilesTypesToSearch" value=".aspx" />

Error:

[WebException: The remote server returned an error: (404) Not Found.]
System.Net.HttpWebRequest.CheckFinalStatus() +674
System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) +139
System.Net.HttpWebRequest.GetResponse() +149
System.Net.WebClient.DownloadData(String address) +151
SearchDotnet.Searchs.UserSearch.GetDynamicFileContent(Page srchFile) +154
SearchDotnet.Searchs.UserSearch.GetFileInfo(String FPath, Page srchFile) +345
SearchDotnet.Searchs.UserSearch.GetInfo(String FPath) +212
SearchDotnet.Searchs.UserSearch.ProcessFile(String FPath) +13
SearchDotnet.Searchs.UserSearch.ProcessDirectory(String targetDirectory) +71
SearchDotnet.Searchs.UserSearch.Search(String targetDirectory) +174
SearchDotnet.SiteSearch.SearchSite(String strSearch) +173
SearchDotnet.SiteSearch.Page_Load(Object sender, EventArgs e) +229
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +731
Stevan Rodrigues

25 Feb 2004 at 01:08

Thanks James for the support and co operation.

I had send the update of my search engine. I know you are busy. So if you want me to format it the way you want I will be pleased to do it. Also if I could be of any help let me know.
stix

24 Feb 2004 at 12:12

Silly me. FOrgot that one. Works fine now. thanks.
James Crowley

24 Feb 2004 at 11:52

Have you got

<add key="EnglishLanguage" value="True" />

or False in your web.config file?
stix

24 Feb 2004 at 11:39

I get this error when trying to run the sample code. Any ideas?

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.FormatException: Input string was not in a correct format.

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace:

[FormatException: Input string was not in a correct format.]
Microsoft.VisualBasic.CompilerServices.DoubleType.Parse(String Value, NumberFormatInfo NumberFormat) +195
Microsoft.VisualBasic.CompilerServices.BooleanType.FromString(String Value) +219

[InvalidCastException: Cast from string "" to type 'Boolean' is not valid.]
Microsoft.VisualBasic.CompilerServices.BooleanType.FromString(String Value) +339
SearchDotnet.Searchs.UserSearch.Search(String targetDirectory) +34
SearchDotnet.SiteSearch.SearchSite(String strSearch) +174
SearchDotnet.SiteSearch.Page_Load(Object sender, EventArgs e) +230
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +725
Stevan Rodrigues

24 Feb 2004 at 00:58

Hi,
The problem is quite basic.
When you place the folder in wwwroot you will need to make that directory ie searchdotnet directory as virtual directory.
srinath_sankar

23 Feb 2004 at 11:47

this is the error that i am getting when i run the same

Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.

Parser Error Message: It is an error to use a section registered as allowDefinition='MachineToApplication' beyond application level. This error can be caused by a virtual directory not being configured as an application in IIS.

Source Error:

Line 34: "Forms", "Passport" and "None"
Line 35: -->
Line 36: <authentication mode="Windows" />
Line 37:
Line 38:

Source File: c:\inetpub\wwwroot\msg10\searchdotnet\web.config Line: 36

--------------------------------------------------------------------------------
Version Information: Microsoft .NET Framework Version:1.1.4322.573; ASP.NET Version:1.1.4322.573
srinath_sankar

23 Feb 2004 at 11:40

Hi I am having problem downloading and running the code any help....I have the searchdotnet directory in my wwwroot andwhen i try to run it it gives me an error saying that the virtual directory missing...and also wanted to know if i can use the same code in my website or do i have to meke changes

thanks
srinath
Stevan Rodrigues

18 Feb 2004 at 07:43

1)In my code I search and filter the data using regular expression. Instead of this you will have to write entire data (not filtered data)to XML file using the following method.

Private Shared Sub WriteXmlToFile(ByVal thisDataSet As DataSet)
If thisDataSet Is Nothing Then
Return
End If

thisDataSet.WriteXml(XMLFile)
End Sub

2)Later you will need to read the xml from file save it to the shared dataset .
Private Shared Function ReadXmlFromFile() As DataSet
' Create a new DataSet.
Dim newDataSet As New DataSet("New DataSet")

' Read the XML document back in.
' Create new FileStream to read schema with.
Dim fsReadXml As New System.IO.FileStream(XMLFile, System.IO.FileMode.Open)

' Create an XmlTextReader to read the file.
Dim myXmlReader As New System.Xml.XmlTextReader(fsReadXml)

' Read the XML document into the DataSet.
newDataSet.ReadXml(myXmlReader)

' Close the XmlTextReader
myXmlReader.Close()
Return newDataSet

End Function
3)For each search you will later need to filter it according to search results. Filter dataset might look like this. Searchs.QueryBuilder.Build(Field, m_searchWords) is the function to build the query.

Private Sub FiterPagesDatset()
Dim strExpr As String
Dim foundRows As DataRow()

Dim Field() As String = {"Title", "Description", "Keywords", "Contents"}

strExpr = Searchs.QueryBuilder.Build(Field, m_searchWords)

foundRows = Searchs.Site.PageDataset.Tables("Pages").Select(strExpr)
FillDataset(foundRows)
End Sub
4)The filtered result store it into another dataset and use it to display results
Hardy Wang

17 Feb 2004 at 09:22

Especially there are large amount of request at same time?
Stevan Rodrigues

17 Feb 2004 at 04:16

When the application is place in the root you may get the following errors.
The remote server returned an error: (401) Unauthorized.
OR
The remote server returned an error: (500) Internal Server Error.

This error is caused because
1)If server returns (401) Unauthorized, the application is unable to read the file beacuse of right access.
2)If server returns (500) Internal Server Error, the page that it was trying to read returned an error. The page that application was trying to read either has an error or requires parameters because of which it returns error

Follow the steps to rectify the error
1)Modify the GetDynamicFileContent method in the UserSearch.vb file

'**********************
'
' GetDynamicFileContent Method
'
' File Content is picked in this method
'
'**********************
Private Sub GetDynamicFileContent(ByVal srchFile As Searchs.Page)
Dim wcMicrosoft As System.Net.WebClient
Dim objUTF8Encoding As UTF8Encoding
Dim strResponse As String

Try
wcMicrosoft = New System.Net.WebClient()
objUTF8Encoding = New UTF8Encoding()
srchFile.Contents = objUTF8Encoding.GetString( _
wcMicrosoft.DownloadData(String.Format("{0}/{1}", Searchs.Site.ApplicationPath, srchFile.Path)))
Catch ex As System.Net.WebException
m_page.Trace.Warn("Error", ex.Message)
End Try

End Sub

2)Rebuild the code
3)In the Web.config ensure that the BarredFolders list is comprehensive
aspnetclient,private,vticnf,vtilog,vtipvt,vtiscript,vtitxt,cgibin,bin,bin,_notes,images,scripts
4)Ensure that the BarredFiles list is comprehensive and contains localstart.asp,iisstart.asp
5)Send me the mail if this issue gets resolved
DLynes

16 Feb 2004 at 05:33

Great code, but I'm concerned about performance issues on large sites or sites with lots of users. Any chance you could post the XML version sometime soon?
Stevan Rodrigues

10 Feb 2004 at 03:23

Thanks James,
Quick work! Code can now be downloaded.
Regards,
Stevan Rodrigues
MCSD.net(Early achiever),
MCAD.Net(Charter Member),MCP
Mobile: 00971 50 5381435
Stevan Rodrigues

09 Feb 2004 at 08:04

Hi James,
Thanks for all.
You have unzipped the searchdotnet.zip file in the parent zip file so the link in html page does not work.
I could not upload the images. So included the images and html file. Since you have added the images we do not require the page and images. But the searchdotnet.zip file in the parent zip is required. So just add the demo code in the zip so the users can download it and test.
Thanks for all.
Regards,
Stevan Rodrigues
MCSD.net(Early achiever),
MCAD.Net(Charter Member),MCP
Mobile: 00971 50 5381435
Stevan Rodrigues

09 Feb 2004 at 07:46

Hi Thushan,
Thanks for the compliment. The inspiration was that there was no code available so write your own. Now who wants to reinvent the wheels.
Chief I have adopted the best object oriented practises and enhanced the code but some people right away rejected it as not useful at Codeproject. It was heart breaking. So I delayed writing my second article on the xml version. But today I have got compliments from a no of places. I am happy that I wrote the article.
Thushan Fernando

09 Feb 2004 at 07:15

Steven,

Great well James is a busy lad(like most of us here) so sometimes things get delayed. If you sent him the code give it a few days for him to sort it out

and yes i was the first to rate it 5/5 here and also the codeproject site

I see you have got some inspiration from WebWizguide's ASP search engine? i used to use that too...
Stevan Rodrigues

09 Feb 2004 at 06:31

Thanks Thushan,
I am happy to know about your comments. It was really encouraging. Code project doesnot have the latest version. I updated the code further and send it to codeproject but no use. Also I uploaded the demo project here. But the admin did n't upload. May be a mistake. Well you can get in touch with me at stevanrin@hotmail.com where I can give you the latest code. Also I have written another version which writes to XML file and reads from it. This code enhances the speed. I still need to document it. I was waiting for a positive feedback for a go ahead.
But don't forget to rate because that tells me where I stand.
James Crowley

09 Feb 2004 at 06:25

The demo code is there - just click the "Download Source Code" link - that zip file includes source code and a demo. (We just can't provide an online demo - you'll have to download it and run it on your own machine)

I've also added images to the tutorial now.

Stevan - if you want a different download to be put up, just email me with it
Stevan Rodrigues

09 Feb 2004 at 06:14

Hi,
Thanks for the comments. Its quite encouraging. I did upload the demo code but realised that the admin has not uploaded it. I will send a quick mail. Also you can get in touch by email at stevanrin@hotmail.com. I will pass on the code.
Regards,
Stevan Rodrigues
MCSD.net(Early achiever),
MCAD.Net(Charter Member),MCP
Mobile: 00971 50 5381435
Thushan Fernando

09 Feb 2004 at 05:45

hi,

I've been looking at this code since it was posted at CodeProject on the 29th (eagerly awaited article i might add)... Steve has put an example here:
http://www.codeproject.com/aspnet/SearchDotnet.asp

I read your other post as well, and i think if its 6000+ pages you should do something about storing common queries in a file(XML?/xSQLx) so that you save CPU cycles...
Emily Grettel

09 Feb 2004 at 05:38

Thanks for the post its really really helpful in needy times.

Would you be able to give us a live demo of this or maybe have a completed version of it uploaded as code so its easier for people in the future?

Thanks alot!
Developer Fusion Bot

01 Jan 1999 at 00:00

This thread is for discussions of Create a Site Search Engine in ASP.NET.

I am a Microsoft Certified Solutions Developer in .Net Architecture (MCSD.Net Early Achiever – one among the first 2500 worldwide), Microsoft Certified Application Developer in .Net – MCAD.Net (Cha...

Ask a question

Related discussion

Using FedEx Web Service to Calculcate Shipping Cost

by bhora123 (4 replies)
Very Urgent regarding deleting the images from a folder

by rameshbandi (2 replies)
Dynamically Generating PDFs in .NET

by nike12 (10 replies)
New style of Javascript used in extenders.

by mittalpa (0 replies)
Not able to launch the web application

by NaseemAhmed (0 replies)

Related podcasts

StackOverflow uses ASP.NET MVC - Jeff Atwood and his technical team

Scott chats with Jeff Atwood of CodingHorror.com and most recently, StackOverflow.com. Jeff and Joel Spolsky and their technical team have created a new class of application using ASP.NET MVC. What works, what doesn't, and how did it all go down?

Library tutorials & articles

Create a Site Search Engine in ASP.NET

UserSearch.vb

Search Method

ProcessDirectory Method

ProcessFile Method

GetInfo Method

Comments

Leave a comment

Related articles

Related discussion

Using FedEx Web Service to Calculcate Shipping Cost

by bhora123 (4 replies)

Very Urgent regarding deleting the images from a folder

by rameshbandi (2 replies)

Dynamically Generating PDFs in .NET

by nike12 (10 replies)

New style of Javascript used in extenders.

by mittalpa (0 replies)

Not able to launch the web application

by NaseemAhmed (0 replies)

Related podcasts

StackOverflow uses ASP.NET MVC - Jeff Atwood and his technical team

Languages

Web Development

Frameworks & Architecture

Other major sections

Library tutorials & articles

Create a Site Search Engine in ASP.NET

UserSearch.vb

Search Method

ProcessDirectory Method

ProcessFile Method

GetInfo Method

Comments

Leave a comment

Related tags

Related articles

Related discussion

Using FedEx Web Service to Calculcate Shipping Cost

by bhora123 (4 replies)

Very Urgent regarding deleting the images from a folder

by rameshbandi (2 replies)

Dynamically Generating PDFs in .NET

by nike12 (10 replies)

New style of Javascript used in extenders.

by mittalpa (0 replies)

Not able to launch the web application

by NaseemAhmed (0 replies)

Related podcasts

StackOverflow uses ASP.NET MVC - Jeff Atwood and his technical team

Languages

Web Development

Frameworks & Architecture

Other major sections