Library tutorials & articles

Create a Site Search Engine in ASP.NET

By Stevan Rodrigues, published on 07 Feb 2004
Page 2 of 8
  1. Introduction
  2. SiteSearch.aspx
  3. Page.vb
  4. UserSearch.vb
  5. FileContent.vb
  6. PagesDataset.vb and CleanHtml.vb
  7. Site.vb and Web.config
  8. How to integrateand Enhancing the code

SiteSearch.aspx

The structure of the Site Search Engine is as follows:

Classes

The ability to define a class and create instances of classes is one of the most important capabilities of any object-oriented language. In the coming section, we see the classes that we have used in the search module.

Class Name Description
SiteSearch Class for the web form where the user can search the site for certain words.
Searches.CleanHtml Class to clean the HTMl content
Searches.FileContent Class to get the content form the HTML File
Searches.Page Class to store data of the pages
Searches.PagesDataset Class to create and store results in dataset
Searches.Site Class to read the configurations of the site
Searches.UserSearch Class to store the search information per user

SiteSearch.aspx

Web Forms are one of the new, exciting features in Microsoft's .NET initiative. SiteSearch.aspx is a web form which is also the start page for the search module.

A Web Forms page consists of a page (ASPX file) and a code behind file (.aspx.cs file or .aspx.vb file). Our web form consists of SiteSearch.aspx and SiteSearch.aspx.vb. I will be treating them simultaneously touching on the main elements of the web form.

ASP.NET is an event-driven programming environment. We will see some event handlers and methods in the coming section.

Page_Load

The server controls are loaded on the Page object and the view state information is available at this point. The Page_Load event checks if sSite is nothing and assigns the Session("Site") variable to it.

      Private Sub Page_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        If IsNothing(sSite) Then
            sSite = Session("Site")
        End If
    End Sub

srchbtn_Click

The search button event is fired when the search button is clicked. Here we put the code to change control settings or display text on the page. Here we check if the search contains text and then call the SearchSite method. DisplayContent() is called to assign values to different controls in the web page. 

    '*********************************************************************
    '
    ' srchbtn_Click event
    '
    ' Add code to this event.
    '
    '*********************************************************************
    Private Sub srchbtn_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles srchbtn.Click
        Dim strSearchWords As String
        'If there is no words entered by the user to search for then dont carryout the file search routine
        pnlSearchResults.Visible = False
        strSearchWords = Trim(Request.Params("search"))

        If Not strSearchWords.Equals("") Then
            Searchs.Site.ApplicationPath = String.Format("http://{0}{1}", Request.ServerVariables("HTTP_HOST"), Request.ApplicationPath)
            sSite = SearchSite(strSearchWords)
            Session("Site") = sSite
            dgrdPages.CurrentPageIndex = 0
            DisplayContent()

        End If
    End Sub

DisplayContent

DisplayContent() is called to assign values to different controls in the web page. The DataGrid content is set by calling the BindDataGrid method. ViewState("SortExpression") is used to store the sort expression. 

    '*********************************************************************
    '
    ' DisplayContent method
    '
    ' The  data is bound to the respective fields.
    '
    '*********************************************************************
    Private Sub DisplayContent()
        If Not IsNothing(sSite.PageDataset) Then
            pnlSearchResults.Visible = True
            lblSearchWords.Text = sSite.SearchWords

            If ViewState("SortExpression") Is Nothing Then
                ViewState("SortExpression") = "MatchCount Desc"
            End If

            BindDataGrid(ViewState("SortExpression"))
            lblTotalFiles.Text = sSite.TotalFilesSearched
            lblFilesFound.Text = sSite.TotalFilesFound
        End If
    End Sub

Search

The main call to the search takes place in this method. UserSearch class which we will cover shortly stores the entire search information and the result of the search. UserSearch object, i.e. srchSite is created and its properties like SearchWords and SearchCriteria assigned. Also srchSite.Search method is called.

      '************************************************************
      '
      ' SearchSite method
      '
      ' The  sSite.PageDataset is used to populate the datagrid.
      '
      '************************************************************
      Private Function SearchSite(ByVal strSearch_
                   As String) As Searchs.UserSearch
        Dim srchSite As Searchs.UserSearch
        srchSite = New Searchs.UserSearch()
        'Read in all the search words into one variable
        srchSite.SearchWords = strSearch

        If Phrase.Checked Then
            srchSite.SearchCriteria = Searchs.SearchCriteria.Phrase
        ElseIf AllWords.Checked Then
            srchSite.SearchCriteria = Searchs.SearchCriteria.AllWords
        ElseIf AnyWords.Checked Then
            srchSite.SearchCriteria = Searchs.SearchCriteria.AnyWords
        End If

        srchSite.Search(Server.MapPath("./"))
        Return srchSite
    End Function

DataGrid

The DataGrid control renders a multi-column, fully templated grid and is by far the most versatile of all data bound controls. Moreover the DataGrid control is the ASP.NET control of choice for data reporting. Hence, I have used it to display the search results. Since the focus of the article is the internal search engine, I will just give a brief overview of the features of the DataGrid used here.

Databinding

Data binding is the process of retrieving data from a source and dynamically associating it to a property of a visual element. Because a DataGrid handles (or at least has in memory) more items simultaneously, you should associate the DataGrid explicitly with a collection of data—that is, the data source.

The content of a DataGrid is set by using its DataSource property. The entire search result is stored in the sSite.PageDataset.Tables("Pages"). Hence the content of the DataGrid is set to dvwPages i.e. sSite.PageDataset.Tables("Pages").DefaultView. BindDataGrid method is called every time the page loads.

      '************************************************************
      '
      ' BindDataGrid method
      '
      ' The  sSite.PageDataset is used to populate the datagrid.
      '
      '************************************************************
      Private Sub BindDataGrid(ByVal strSortField As String)
            Dim dvwPages As DataView
            dvwPages = sSite.PageDataset.Tables("Pages").DefaultView
            dvwPages.Sort = strSortField
            dgrdPages.DataSource = dvwPages
            dgrdPages.DataBind()
      End Sub

The control has the ability to automatically generate columns that are based on the structure of the data source. Auto-generation is the default behavior of DataGrid, but you can manipulate that behavior by using a Boolean property named AutoGenerateColumns. Set the property to False when you want the control to display only the columns you explicitly add to the Columns collection. Set it to True (the default) when you want the control to add as many columns as is required by the data source. Auto-generation does not let you specify the header text, nor does it provide text formatting. Hence, here I have set it to False. You typically bind columns using the <columns> tag in the body of the <asp:datagrid> server control.

 <Columns>
  <asp:TemplateColumn>
    <ItemTemplate>
      <%# DisplayTitle(Container.DataItem( "Title" ), _
                Container.DataItem( "Path" )) %>
      <br>
      <%# Container.DataItem( "Description" ) %>
      <br>
      <span class="Path">
        <%# String.Format("{0} - {1}kb", DisplayPath(Container.DataItem( "Path" )) ,Container.DataItem( "Size" ))%>
      </span>
      <br>
      <br>
    </ItemTemplate>
  </asp:TemplateColumn>
</Columns>

DisplayTitle method and DisplayPath method are used to display customized information to the columns in the DataGrid.

      '****************************************
      '
      ' DisplayTitle method
      '
      ' Display title of searched pages 
      '
      '****************************************
      Protected Function DisplayTitle(ByVal Title _ 
              As String, ByVal Path As String) As String
            Return String.Format("<A href="{1}">{0}</A>", Title, Path)
      End Function

      '****************************************
      '
      ' DisplayPath method
      '
      ' Path of the file is returned 
      '
      '****************************************
      Protected Function DisplayPath(ByVal Path As String) As String
            Return String.Format("{0}{1}/{2}", _ 
              Request.ServerVariables("HTTP_HOST"), _ 
              Request.ApplicationPath, Path)
      End Function
Pagination

Unlike the DataList control, the DataGrid control supports data pagination, that is, the ability to divide the displayed data source rows into pages. The size of our data source easily exceeds the page real estate. So to preserve scalability on the server and to provide a more accessible page to the user, you display only a few rows at a time. To enable pagination of the DataGrid control, you need to tell the control about it. You do this through the AllowPaging property.

The pager bar is an interesting and complimentary feature offered by the DataGrid control to let users easily move from page to page. The pager bar is a row displayed at the bottom of the DataGrid control that contains links to available pages. When you click any of these links, the control automatically fires the PageIndexChanged event and updates the page index accordingly. dgrdPages_PageIndexChanged is called when the page index changes.

      '*****************************************************************
      '
      ' dgrdPages_PageIndexChanged event
      '
      ' The CurrentPageIndex is Assigned the page index value.
      ' The datagrid is then populated using the BindDataGrid function.
      '
      '*****************************************************************
      Protected Sub dgrdPages_PageIndexChanged(ByVal s As Object, _ 
          ByVal e As DataGridPageChangedEventArgs) _ 
          Handles dgrdPages.PageIndexChanged
      dgrdPages.CurrentPageIndex = e.NewPageIndex
      DisplayContent()
      End Sub

You control the pager bar by using the PagerStyle property’s Mode attribute. Values for the Mode attribute come from the PagerMode enumeration. Here we have chosen a detailed series of numeric buttons, each of which points to a particular page.

<PagerStyle CssClass="GridPager" Mode="NumericPages"></PagerStyle>
Sorting

The DataGrid control does not actually sort rows, but it provides good support for sorting as long as the sorting capabilities of the underlying data source are adequate. The data source is always responsible for returning a sorted set of records based on the sort expression selected by the user through the DataGrid control’s user interface. The built-in sorting mechanism is triggered by setting the AllowSorting property to True.

dgrdPages_SortCommand is called to sort the DataGrid. The SortCommand event handler knows about the sort expression through the SortExpression property, which is provided by the DataGridSortCommandEventArgs class. In our code, the sort information is persisted because it is stored in a slot in the page’s ViewState collection.

Note: In my pages, I have disabled the header but if the header is shown, you can use it to sort the DataGrid.

      '*****************************************************************
      '
      ' dgrdAdditionalItems_SortCommand event
      '
      ' The ViewState( "SortExpression" ) is Assigned
      ' the sort expression value.
      ' The datagrid is then populated using the BindDataGrid function.
      '
      '*****************************************************************
      Protected Sub dgrdPages_SortCommand(ByVal s As Object, _ 
             ByVal e As DataGridSortCommandEventArgs) _ 
             Handles dgrdPages.SortCommand
            ViewState("SortExpression") = e.SortExpression
            DisplayContent()
      End Sub

Comments

  1. John Loser
    28 May 2008 at 04:09

    hello all

    i am trying hard to integrate the search engine code but have failed, my site is on local lan server with the root folder as a virtual folder , my site is in c# , i have placed the contents in the folder named searchdotnetc , but the code dosnt works, i modified the code made the class partial for the cs file and also tried placing the contents folder in the app code , but i still parsers only static pages and that to only in the searchdotnetc folder for the dynamic pages it gives  the error

     


    Line 273:			wcMicrosoft = new System.Net.WebClient();
Line 274:			objUTF8Encoding = new UTF8Encoding();
Line 275:			srchFile.Contents = objUTF8Encoding.GetString(
Line 276:				wcMicrosoft.DownloadData(String.Format("{0}/{1}", Searchs.Site.ApplicationPath, srchFile.Path)));
Line 277:		}

    Source File: v:\SITENAMEwww\App_Code\components\UserSearch.cs    Line: 275

    Stack Trace: 

    [WebException: The remote server returned an error: (404) Not Found.]
   System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request) +338
   System.Net.WebClient.DownloadData(Uri address) +181
   System.Net.WebClient.DownloadData(String address) +26
   Searchs.UserSearch.GetDynamicFileContent(Page srchFile) in v:\SITENAME\www\App_Code\components\UserSearch.cs:275

    please help .....

     

     

  2. shaka
    25 Jun 2007 at 19:01
    Do you have a version of the Search Engine project, for ASP.NET 2.0, that works well with Master Pages?
  3. sree2000
    02 Jun 2006 at 17:53

    Hello,

         I have down loaded the SearchDotNet.zip file from the code project. I am using VS2005, and asp.net 2.0  the files are not getting converted successfully. Can you provide me a solution to integrate this search facility into my web application .

  4. Deepthi P
    31 Jan 2006 at 07:25
    when i used ur solution to implement search engine for my website it is  searching only words .
    for example if hai is given it is finding some files having "hai" but if h is given it is showing " 0 found"
    please suggest me one solution
  5. Cassius Arulkumaran
    18 Jan 2006 at 21:47

    you can use text copy to copy images and text objects inside of SQL server
    http://www.mssqlcity.com/Articles/KnowHow/Textcopy.htm

  6. kody
    03 Nov 2005 at 23:07
    Did you find it ? Because I am also searching for something like that.
  7. trobin7
    04 Oct 2005 at 15:37

    It seems I have this error as well.  Do I need to give write access to the root directory or access to another file or something else?


    As a test, I've given the "Everyone" full access to the root, but doesn't help


    Thanks ..Troy

  8. Stevan Rodrigues
    13 Sep 2005 at 08:53

    The error is related to the permissions. You will need to check the permissions and ensure that the application has permissions expecially  System.Net.WebClient

  9. Najeem M Illyas
    11 Sep 2005 at 13:58

    Hi I'm also facing a similar problem with the search program. I'm getting error like 404. Anybody tell me how to fix the problem, I've configured the web.config file correctly.

  10. oramike
    18 May 2005 at 15:10

    hi, i have an SQL Server database with documents from applications (ms word, excel, etc). i now need to retrieve these documents in their format but don't know how to go about it.


    can anyone lend a hand, please!

  11. oramike
    15 Apr 2005 at 15:30
    Hi James, thanks for the help. I finally got it.

  12. James Crowley
    14 Apr 2005 at 23:17
    In IIS, you either need to make the virtual directory an "application". To do this, right click on the virtual directory (in IIS admin), click properties, and click the "Create" button next to "application name".

    If you don't have an application configured, then there's a set of configuration options in web.config that aren't supported - hence it saying "It is an error to use a section registered as allowDefinition='MachineToApplication' beyond application level.". If you remove the <authentication> tag entirely, then this should work (or possibly it will complain about another unsupported tag )
  13. oramike
    14 Apr 2005 at 19:44
    When i tried to run the code, i get this error message:

    Configuration Error.

    Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.

    Parser Error Message: It is an error to use a section registered as allowDefinition='MachineToApplication' beyond application level. This error can be caused by a virtual directory not being configured as an application in IIS.

    Source Error:


    Line 34:           "Forms", "Passport" and "None"
    Line 35:     -->
    Line 36:     <authentication mode="Windows" />
    Line 37:
    Line 38:


    Source File: c:\inetpub\wwwroot\SearchDotnet\web.config    Line: 36.


    I changed the authentication mode to "None", but i still get the same error message.

    Is there anything i should do that i have not yet done?


  14. pax
    04 Feb 2005 at 01:23

    i'm trying to search in an asp file but it doesn't work, i'm not getting the error 500, it's solved but i still not searching on the asp file, it appears like there's no match on that file but there's a match. what can i do?

  15. tiglathpileser
    09 Dec 2004 at 10:06

    we make a search engine that we are in the process of porting to .net  Get in touch with me if you are interested in doing some beta testing.  Office document indexing is quite tricky with a lack of documentation available and pdf documents store information in a variety of strange ways.  contact me: paddy phdcc.com

  16. Hugmup
    07 Dec 2004 at 18:39

    I am not a Visual Studio developer, but I have devleped about 8 ASPX projects with a notepad called ScITE for my web site and I've compiled my DLLs with the command-line compiler.


    I downloaded the source files. I put SearchSite.aspx in the wwwroot\website directory on my local IIS server, I put the DLL in wwwroot\website\bin, and I put the web.config file in wwwroot\website. Then I configured web.config according to the instructions, modified the HTML in the ASPX file so that it matches my web site, and changed the name of SiteSearch.aspx to zsearch.aspx to match my file naming scheme.


    It works beautifully on my local IIS server, but it does not work on the production server. The error is Service Unavailable.


    This is not useful.

  17. Stevan Rodrigues
    03 Dec 2004 at 21:31

    My code searches for files that store content as text or removes text from html. It is not made to search for pdf and word documents.

  18. jackthecat64
    03 Dec 2004 at 13:44

    Hi. I read your query to Stevan about dealing with .pdf files in his search engine (searchdotnet).  I don't think he replied to you, right?


    Did you solve this problem?  I'm am having the same issue.


    Thanks in advance for any help you can offer.


    Beth

  19. musa7290
    28 Sep 2004 at 19:31

    I would highly appreciate if someone points me some sample code for simple search engine\crawler\robot that can be used\extended to gather news headlines from news websites.


    Basically I am highly interested to implement a small live new page like news.google.com
    Thanks in Advance
    Musa


  20. fpga
    17 Aug 2004 at 08:19

    Hi,
    I would integrate your solution in my web...but , something I don't understund...
    you explain that we have to "2) Place the SiteSearch.aspx and web.config  in the root  of the application"


    but , I allready have a web.config , for my application.
    ...
    can you give me more information about it ?


    thanks

  21. Bob Reichard
    16 Aug 2004 at 08:30

    I have your Sitesearch code working fine on my development system and development server. However, on my live server I receive the following error. But if I log into the server through terminal and acces the site in a browser from that box it works fine. Any idea why this is happening? Permissions all seem to be fine.


    The underlying connection was closed: Unable to connect to the remote server.
    An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.
    [WebException: The underlying connection was closed: Unable to connect to the remote server.]
      System.Net.HttpWebRequest.CheckFinalStatus() +673
      System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) +140
      System.Net.HttpWebRequest.GetResponse() +149
      System.Net.WebClient.DownloadData(String address) +151
      clearweld.Searchs.UserSearch.GetDynamicFileContent(Page srchFile)
      clearweld.Searchs.UserSearch.GetFileInfo(String FPath, Page srchFile)
      clearweld.Searchs.UserSearch.GetInfo(String FPath)
      clearweld.Searchs.UserSearch.ProcessFile(String FPath)
      clearweld.Searchs.UserSearch.ProcessDirectory(String targetDirectory)
      clearweld.Searchs.UserSearch.Search(String targetDirectory)
      clearweld.SiteSearch.SearchSite(String strSearch)
      clearweld.SiteSearch.Page_Load(Object sender, EventArgs e)
      System.Web.UI.Control.OnLoad(EventArgs e) +67
      System.Web.UI.Control.LoadRecursive() +35
      System.Web.UI.Page.ProcessRequestMain() +731

  22. Ilkin E Jamalli
    27 Jul 2004 at 23:17
    I have installed on c:\wwwroot\search  folder.
    It works fine but it search only on c:\wwwroot\search\sites  folder. How can I make it to search in c:\wwwroot\
    Can you please suggest ?
  23. Stevan Rodrigues
    23 Jun 2004 at 22:15

    Hi,
    I have sent the updates but the admin here is too too busy to do changes. So my latest code can be found at
    http://www.codeproject.com/aspnet/SearchDotnet.asp. I have uploaded the c# version there. The C# version is older version If you want to use VB.net code and since you have webmatrix then


    1)You can just use the dll of vb.net search engine and place it in the bin of your project


    2) Use the aspx pages in your c sharp project


    3)Convert the code behind vb pages to c sharp



    Steve

  24. Sri Koerniwati
    23 Jun 2004 at 10:06

    Hi Steve,


    I am beginner asp.net programmer. I am developing website using webmatrix, I use C#. I create a folder for all my pages then I created a virtual directory in IIS. I download your search engine vb code, I copy searchdotnet folder to my website folder then I run SiteSearch.aspx. The page was loading and when I tried to search words that are in my pages, I got this message: 0 Files found. I tried to put one of my pages in Sites folder under SearchDotNet Folder and tried again to search word on my pages and I still got the same message: 0 Files found.


    Could you help me with this ? Where should I put my pages so when I run the SiteSearch.aspx, I can find words that are on my pages.... how to convert VB.net code to C#?


    Thank you so much for helping me....

  25. Stevan Rodrigues
    21 Jun 2004 at 00:10

    Hi,
    1)You can just use the dll of vb.net search engine and place it in the bin of your project


    2) Use the aspx pages in your c sharp project


    3)Convert the code behind vb pages to c sharp


    4)My latest code can be found at
    http://www.codeproject.com/aspnet/SearchDotnet.asp


    Steve

  26. Emily Grettel
    20 Jun 2004 at 23:18
    Hi Steve!

    Just wondering, if you knew how much of a problem it would be if we integrated your VB.NET search tool in our C# web application for our intranet... theres roughly 400 pages (dynamic and static) but i'm not sure whether things will cause problems because ours is written in C# and the search engine is VB.NET.

    Also, we would be having it in the root (Eg. /search.aspx) so would this be a problem? or do we need to manually convert all to C# before we try and integrate it?

    Thanks and congratulations on a FANTASTIC and truely excellent piece of work:-)

  27. Emily Grettel
    20 Jun 2004 at 23:18
    Hi Steve,

    Just to let you know I posted a msg about your script here:
    http://www.developerfusion.com/forums/topic.aspx?id=21918
  28. Alan
    09 Jun 2004 at 01:43

    Great code stevan....!!


    Was just wondering if you had any ideas about working with PDF's and word documents


    I am using the XML version to search an entire intranet. The site contains a large number of pdfs and documents that have embedded graphics.


    The search functionality works perfectly. However, the xml file contains code that represents the graphic part of these documents. Therefore when a search is completed, the content field returns meaingless information:



    Search on Marketing returns:


    Title: Marketing
    Content: PDF 1 3 6 0 obj endobj xref 6 36 0000000016 00000 n 0000001064 00000 n 0000001368 00000 n 0000001572 00000 n 0000001730 00000 n 0000002139 00000 n 000000266    



    Any ideas how this could be filtered out...???

  29. rogeriogr
    14 May 2004 at 10:48
    Hi folks,

    I am trying the SiteSearch script in my personal web page and it is working well except by the fact that the portuguese accents are not being showed on the search results.

    For example, when showing the results ú appears as uacute; ç appears as ccedil.

    Thanks in advance.
  30. Stevan Rodrigues
    07 May 2004 at 22:33

    The Latest Code and images can be found at http://www.codeproject.com/aspnet/SearchDotnet.asp.

  31. Stepan Kazakov
    22 Apr 2004 at 10:22

    I got your updated version and when i try to build it i encouter this problem. Any idea why?




    Thanks

  32. Stevan Rodrigues
    08 Apr 2004 at 04:37

    Hi,
    I have updated my article n sent to the admin but there has been a delay.Just try this
    http://www.freewebs.com/stevanrodrigues/searchdotnet.html
    Regards,
    Stevan

  33. Manoj Kumar
    29 Mar 2004 at 07:37

    Dear Sir,


    This is manoj and using your site search engine
    and facing so many errors while using it.
    can you give me the process from A-Z how to use it?


    Thanks & Regards
    manoj(mnj_secret@yahoo.com)

  34. asplover
    20 Mar 2004 at 12:44
    hello,firend:
      i come form china.i appreciate your site search engine very well. but i encount a problem that it can not search chinese word. please tell me how to solve this problem.
                                            thank u very much!
                                                                                             asplover
                                                                                               3.20
  35. Simone75
    09 Mar 2004 at 03:38

    Thank you Stevan,
    Now all work correctly.
    I use your dll and it works fine.
    I've a c# web project so i can't integrate your vb.net project with mine using your guidelines.
    In a later version of your project think about configuration of the root in wich begin the search; i think it is helpful.
    Thank you very much.
    Bye.

  36. Stevan Rodrigues
    09 Mar 2004 at 01:13

    Well just go to the http://www.freewebs.com/stevanrodrigues/SearchDotnet.html where you will get the latest update. Cick on the third link ie..
    Download latest version of demo project (Visual studio.net not required)
    1) Place the SearchDotnet.dll in the bin folder in the root of the application eg: http:\localhost\abitaria.
    2) Place the SiteSearch.aspx and web.config in the root  of the application eg: http:\localhost\abitaria.


    If you still want to integrate the code and not just dll in your page then
    1)Since you already have a project in visual studio .net open the old project(vb.net web application) which is pointing to the root of the application eg: http:\localhost\abitaria.
    2)Place the sitesearch.aspx in the directory "abitaria".
    3)Place the components folder  in root "abitaria" or any folder within "abitaria"
    4)Build the code.



    Regards,


    Stevan Rodrigues

  37. Simone75
    08 Mar 2004 at 04:43

    I'm sorry but i've problems yet.
    1)I'm not able to open In visual studio .net a new project(vb.net web application) pointing to the root of the application eg: http:\localhost\abitaria because visual studio .net tell me that already exists a project pointing in that root.
    2)I tried, also, to  ONLY copy sitesearch.aspx and Components directory in root Abitaria but when i try to load the aspx page this error is shown:
    Description: An error occurred during the parsing of a resource required to service this request. Please review the following specific parse error details and modify your source file appropriately.


    Parser Error Message: Could not load type 'SearchDotnet.SiteSearch'.


    Source Error:



    Line 1:  <%@ Page Language="vb" Trace="False" AutoEventWireup="false" Codebehind="SiteSearch.aspx.vb" Inherits="SearchDotnet.SiteSearch" debug="false" %>
    Line 2:  <HTML>
    Line 3:      <HEAD>



    Source File: C:\Inetpub\wwwroot\Abitaria\sitesearch.aspx    Line: 1


    So i think i must change the code in sitesearch.aspx because it inherits from your SearchDotnet project


    I like your SearchDotnet application and i want to use it in my web sites but i need more help.
    Could you help me again?
    Thank you.
    Simone

  38. Stevan Rodrigues
    07 Mar 2004 at 00:46
    Hi,

    Do you have visual studio .net? Do you have the .Net framework?

    1)In visual studio .net open a new project(vb.net web application) pointing to the root of the application eg: http:\\localhost\abitaria.
    2)Place the sitesearch.aspx in the directory "abitaria".
    3)Place the components folder  in root "abitaria" or any folder within "abitaria"
    4)Build the code.

    There is a work around do all this without visual studio .net. I have a solution without visual studio send me an email and I will let you know.

    Regards,

    Stevan Rodrigues
  39. Simone75
    05 Mar 2004 at 10:46
    Hello,
    i want to knows how i can integrate your site search web application in my web application.
    I have a virtual directory called "abitaria", you use your virtual directory called "searchdotnet".
    I want sitesearch.aspx in my virtual directory and i want the search in my root "http:\\localhost\abitaria" not in "http:\\localhost\searchdotnet\site".
    Can you help me???
    thank you
    email me
  40. Stevan Rodrigues
    29 Feb 2004 at 05:53

    I have sent the latest update to admin till then you can try the code at
    http://www.codeproject.com/aspnet/SearchDotnet.asp
    Also you can get in touch with me by email.

  41. Stevan Rodrigues
    26 Feb 2004 at 00:36

    Well just run a search for 'error' you will get the page listed there. This is because your page may require some parameter(querystring or form) or cannot be read directly. Try calling from the web using that path in address of your browser.

  42. Dennis St. John
    26 Feb 2004 at 00:20

    I have been successful in running this app in a virtual directory in IIS (i.e. /SearchDotNet), however, what if I want to search the files in the root directory as well?  I copied the app to the root and can't seem to get it to work as it does within a virtual directory underneath of the root.  Any help or suggestions would be greatly appreciated.


    Thanks!


    Dennis - PC1, Inc.

  43. Joshua Stengel
    25 Feb 2004 at 13:25

    I made a modification as show in another post
    and that got rid of the error but it still doesn't
    search the dynamic content.  I am investigating...


    See post for 500 and 401 errors:
    http://www.developerfusion.com/forums/topic-19327

  44. Joshua Stengel
    25 Feb 2004 at 12:16

    When searching dynamic .aspx pages I get a 404 error.
    Any thoughts?


    I have the following lines in my web.config file and the
    engine runs fine in directories that do not contain
    dynamic content.


    web.config lines:


            <add key="FilesTypesToSearch" value=".aspx" />
            <add key="DynamicFilesTypesToSearch" value=".aspx" />


    Error:


    [WebException: The remote server returned an error: (404) Not Found.]
      System.Net.HttpWebRequest.CheckFinalStatus() +674
      System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) +139
      System.Net.HttpWebRequest.GetResponse() +149
      System.Net.WebClient.DownloadData(String address) +151
      SearchDotnet.Searchs.UserSearch.GetDynamicFileContent(Page srchFile) +154
      SearchDotnet.Searchs.UserSearch.GetFileInfo(String FPath, Page srchFile) +345
      SearchDotnet.Searchs.UserSearch.GetInfo(String FPath) +212
      SearchDotnet.Searchs.UserSearch.ProcessFile(String FPath) +13
      SearchDotnet.Searchs.UserSearch.ProcessDirectory(String targetDirectory) +71
      SearchDotnet.Searchs.UserSearch.Search(String targetDirectory) +174
      SearchDotnet.SiteSearch.SearchSite(String strSearch) +173
      SearchDotnet.SiteSearch.Page_Load(Object sender, EventArgs e) +229
      System.Web.UI.Control.OnLoad(EventArgs e) +67
      System.Web.UI.Control.LoadRecursive() +35
      System.Web.UI.Page.ProcessRequestMain() +731

  45. Stevan Rodrigues
    25 Feb 2004 at 01:08

    Thanks James for the support and co operation.


    I had send the update of my search engine. I know you are busy. So if you want me to format it the way you want I will be pleased to do it. Also if I could be of any help let me know.

  46. stix
    24 Feb 2004 at 12:12

    Silly me. FOrgot that one. Works fine now. thanks.

  47. James Crowley
    24 Feb 2004 at 11:52

    Have you got


    <add key="EnglishLanguage" value="True" />


    or False in your web.config file?

  48. stix
    24 Feb 2004 at 11:39

    I get this error when trying to run the sample code. Any ideas?


    Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.


    Exception Details: System.FormatException: Input string was not in a correct format.


    Source Error:


    An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.  


    Stack Trace:



    [FormatException: Input string was not in a correct format.]
      Microsoft.VisualBasic.CompilerServices.DoubleType.Parse(String Value, NumberFormatInfo NumberFormat) +195
      Microsoft.VisualBasic.CompilerServices.BooleanType.FromString(String Value) +219


    [InvalidCastException: Cast from string "" to type 'Boolean' is not valid.]
      Microsoft.VisualBasic.CompilerServices.BooleanType.FromString(String Value) +339
      SearchDotnet.Searchs.UserSearch.Search(String targetDirectory) +34
      SearchDotnet.SiteSearch.SearchSite(String strSearch) +174
      SearchDotnet.SiteSearch.Page_Load(Object sender, EventArgs e) +230
      System.Web.UI.Control.OnLoad(EventArgs e) +67
      System.Web.UI.Control.LoadRecursive() +35
      System.Web.UI.Page.ProcessRequestMain() +725



  49. Stevan Rodrigues
    24 Feb 2004 at 00:58
    Hi,
    The problem is quite basic.
    When you place the folder in wwwroot you will need to make that directory ie searchdotnet directory as virtual directory.
  50. srinath_sankar
    23 Feb 2004 at 11:47
    this is the error that i am getting when i run the same

    Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.

    Parser Error Message: It is an error to use a section registered as allowDefinition='MachineToApplication' beyond application level. This error can be caused by a virtual directory not being configured as an application in IIS.

    Source Error:


    Line 34:           "Forms", "Passport" and "None"
    Line 35:     -->
    Line 36:     <authentication mode="Windows" />
    Line 37:
    Line 38:


    Source File: c:\inetpub\wwwroot\msg10\searchdotnet\web.config    Line: 36


    --------------------------------------------------------------------------------
    Version Information: Microsoft .NET Framework Version:1.1.4322.573; ASP.NET Version:1.1.4322.573
  51. srinath_sankar
    23 Feb 2004 at 11:40
    Hi I am having problem downloading and running the code any help....I have the searchdotnet directory in my wwwroot andwhen i try to run it it gives me an error saying that the virtual directory missing...and also wanted to know if i can use the same code in my website or do i have to meke changes

    thanks
    srinath
  52. Stevan Rodrigues
    18 Feb 2004 at 07:43

    1)In my code I search and filter the data using regular expression. Instead of this you will have to write entire data (not filtered data)to XML file using the following method.


         Private Shared Sub WriteXmlToFile(ByVal thisDataSet As DataSet)
               If thisDataSet Is Nothing Then
                   Return
               End If
               
               thisDataSet.WriteXml(XMLFile)
           End Sub



    2)Later you will need to read the xml from file save it to the shared dataset .
           Private Shared Function ReadXmlFromFile() As DataSet
               ' Create a new DataSet.
               Dim newDataSet As New DataSet("New DataSet")


               ' Read the XML document back in.
               ' Create new FileStream to read schema with.
               Dim fsReadXml As New System.IO.FileStream(XMLFile, System.IO.FileMode.Open)


               ' Create an XmlTextReader to read the file.
               Dim myXmlReader As New System.Xml.XmlTextReader(fsReadXml)


               ' Read the XML document into the DataSet.
               newDataSet.ReadXml(myXmlReader)


               ' Close the XmlTextReader
               myXmlReader.Close()
               Return newDataSet


           End Function
    3)For each search you will later need to filter it according to search results. Filter dataset might look like this. Searchs.QueryBuilder.Build(Field, m_searchWords) is the function to build the query.



          Private Sub FiterPagesDatset()
               Dim strExpr As String
               Dim foundRows As DataRow()


               Dim Field() As String = {"Title", "Description", "Keywords", "Contents"}


               strExpr = Searchs.QueryBuilder.Build(Field, m_searchWords)


             
               foundRows = Searchs.Site.PageDataset.Tables("Pages").Select(strExpr)
               FillDataset(foundRows)
           End Sub
    4)The filtered result store it into another dataset and use it to display results

  53. Hardy Wang
    17 Feb 2004 at 09:22

    Especially there are large amount of request at same time?

  54. Stevan Rodrigues
    17 Feb 2004 at 04:16

    When the application is place in the root you may get the following errors.
    The remote server returned an error: (401) Unauthorized.
    OR
    The remote server returned an error: (500) Internal Server Error.


    This error is caused because
    1)If server returns (401) Unauthorized, the application is unable to read the file beacuse of right access.
    2)If server returns (500) Internal Server Error, the page that it was trying to read returned an error. The page that application was trying to read either has an error or requires parameters because of which it returns error


    Follow the steps to rectify the error
    1)Modify the GetDynamicFileContent method in the UserSearch.vb file


           '**********************
           '
           ' GetDynamicFileContent Method
           '
           ' File Content is picked in this method
           '
           '    **********************
           Private Sub GetDynamicFileContent(ByVal srchFile As Searchs.Page)
               Dim wcMicrosoft As System.Net.WebClient
               Dim objUTF8Encoding As UTF8Encoding
               Dim strResponse As String


               Try
                   wcMicrosoft = New System.Net.WebClient()
                   objUTF8Encoding = New UTF8Encoding()
                   srchFile.Contents = objUTF8Encoding.GetString( _
                   wcMicrosoft.DownloadData(String.Format("{0}/{1}", Searchs.Site.ApplicationPath, srchFile.Path)))
               Catch ex As System.Net.WebException
                   m_page.Trace.Warn("Error", ex.Message)
               End Try


           End Sub


    2)Rebuild the code
    3)In the Web.config ensure that the BarredFolders list is comprehensive
    aspnetclient,private,vticnf,vtilog,vtipvt,vtiscript,vtitxt,cgibin,bin,bin,_notes,images,scripts
    4)Ensure that the BarredFiles list is comprehensive and contains localstart.asp,iisstart.asp
    5)Send me the mail if this issue gets resolved

  55. DLynes
    16 Feb 2004 at 05:33

    Great code, but I'm concerned about performance issues on large sites or sites with lots of users. Any chance you could post the XML version sometime soon?

  56. Stevan Rodrigues
    10 Feb 2004 at 03:23

    Thanks James,
    Quick work! Code can now be downloaded.
    Regards,
    Stevan Rodrigues
    MCSD.net(Early achiever),
    MCAD.Net(Charter Member),MCP
    Mobile: 00971 50 5381435

  57. Stevan Rodrigues
    09 Feb 2004 at 08:04

    Hi James,
    Thanks for all.
    You have unzipped the searchdotnet.zip file in the parent zip file so the link in html page does not work.
    I could not upload the images. So included  the images and html file. Since you have added the images we do not require the page and images. But the searchdotnet.zip file in the parent zip is required. So just add the demo code in the zip so the users can download it and test.
    Thanks for all.
    Regards,
    Stevan Rodrigues
    MCSD.net(Early achiever),
    MCAD.Net(Charter Member),MCP
    Mobile: 00971 50 5381435

  58. Stevan Rodrigues
    09 Feb 2004 at 07:46

    Hi Thushan,
    Thanks for the compliment. The inspiration was that there was no code available so write your own. Now who wants to reinvent the wheels.  
    Chief I have adopted the best object oriented practises and enhanced the code but some people right away rejected it as not useful at Codeproject. It was heart breaking. So I delayed writing my second article on the xml version. But today I have got compliments from a no of places. I am happy that I wrote the article.

  59. Thushan Fernando
    09 Feb 2004 at 07:15

    Steven,


    Great well James is a busy lad(like most of us here) so sometimes things get delayed. If you sent him the code give it a few days for him to sort it out


    and yes i was the first to rate it 5/5 here and also the codeproject site


    I see you have got some inspiration from WebWizguide's ASP search engine? i used to use that too...

  60. Stevan Rodrigues
    09 Feb 2004 at 06:31

    Thanks Thushan,
    I am happy to know about your comments. It was really encouraging. Code project doesnot have the latest version. I updated the code further and send it to codeproject but no use. Also I uploaded the demo project here. But the admin did n't upload. May be a mistake. Well you can get in touch with me at stevanrin@hotmail.com where I can give you the latest code. Also I have written another version which writes to XML file and reads from it. This code enhances the speed. I still need to document it. I was waiting for a positive feedback for a go ahead.
    But don't forget to rate because that tells me where I stand.

  61. James Crowley
    09 Feb 2004 at 06:25

    The demo code is there - just click the "Download Source Code" link - that zip file includes source code and a demo. (We just can't provide an online demo - you'll have to download it and run it on your own machine)


    I've also added images to the tutorial now.


    Stevan - if you want a different download to be put up, just email me with it

  62. Stevan Rodrigues
    09 Feb 2004 at 06:14

    Hi,
    Thanks for the comments. Its quite encouraging. I did upload the demo code but realised that the admin has not uploaded it. I will send a quick mail. Also you can get in touch by email at stevanrin@hotmail.com. I will pass on the code.
    Regards,
    Stevan Rodrigues
    MCSD.net(Early achiever),
    MCAD.Net(Charter Member),MCP
    Mobile: 00971 50 5381435

  63. Thushan Fernando
    09 Feb 2004 at 05:45

    hi,


    I've been looking at this code since it was posted at CodeProject on the 29th (eagerly awaited article i might add)... Steve has put an example here:
    http://www.codeproject.com/aspnet/SearchDotnet.asp


    I read your other post as well, and i think if its 6000+ pages you should do something about storing common queries in a file(XML?/xSQLx) so that you save CPU cycles...


  64. Emily Grettel
    09 Feb 2004 at 05:38

    Thanks for the post its really really helpful in needy times.


    Would you be able to give us a live demo of this or maybe have a completed version of it uploaded as code so its easier for people in the future?


    Thanks alot!

  65. Developer Fusion Bot
    01 Jan 1999 at 00:00

    This thread is for discussions of Create a Site Search Engine in ASP.NET.

Leave a comment

Sign in or Join us (it's free).

Related tags

asp.net
Stevan Rodrigues I am a Microsoft Certified Solutions Developer in .Net Architecture (MCSD.Net Early Achiever – one among the first 2500 worldwide), Microsoft Certified Application Developer in .Net – MCAD.Net (Cha...
AddThis

Related articles

Ask a question

Related discussion

Related podcasts

Languages

Web Development

Frameworks & Architecture

Other major sections

Cancel
We'd love to hear what you think! Submit ideas or give us feedback