Comment Installer Sphider

Posted on
  • 11/09/13--11:17: How can I search for particular user documents
  • Hi
    I need Sphider to index all documents but when searching
    I need to somehow pass a 'user' parameter to make sure only
    current user's owned documents are searched.
    Any idea how can I go about this?
    Thanks,

  • 11/10/13--02:55: Re: How can I search for particular user documents
  • Organize the documents for each user in a separate subfolder of your site.
    Example:
    your_domain.com/user1/
    your_domain.com/user2/
    your_domain.com/user3/
    etc.
    Now add all these URLs inclusive the subfolders as URL to be indexed into the Admin backend of your Sphider installation.
    Next open the 'Category' menu in your Admin backend and define an individual category (name) for each of your user.
    Then open the 'Edit' menu in 'Options' of the first URL. Find the field 'URLs must include' and enter
    /user1
    into this field. Additionally active the checkbox1 as the valid category for this URL.
    Continue with the second URL and add
    /user2
    into the field 'URLs must include'. Also active the checkbox2 as the valid category for this URL.
    Continue for all your users.
    Afterwards index all URLs placed in your Admin backend. The indexer will prepare individual keywords for each user, as only the according subfolder is indexed, controlled by the 'URLs must include' rule.
    Now the users of your search engine may define a category in the search form. Selecting one category will force the search algorithm to find only those keywords, belonging to one user/category.
    In case not all users are allowed to search in all categories, you will have to prepare individual search forms for each user. Also you will have to pass the category by a hidden form field to the search script.
    Tec

  • 11/12/13--07:21: Searching from results page
  • Greetings all,
    I run the website www.screenmachine.com and we've had Sphider on there for some time without much issue. I noticed this morning though that if I perform a search and go to the results page, if I search from that results page that the new search doesn't submit.
    Example results page: [www.screenmachine.com]
    It's acting like the submit button just doesn't work and I'm hitting a brick wall. Searching from the main website (not from a search results page) works perfectly.
    Any thoughts would be greatly appreciated!

  • 11/12/13--07:32: Re: Searching from results page
  • And.. It appears that the back button was the culprit. I removed it and it seems to work fine?

    Still, that's a minor quibble; both the first At the Village Vanguard and Vol. The great jazz trio at the village vanguard rarities.

  • 11/13/13--07:51: Re: How can I search for particular user documents
  • Don't know, why your last posting got lost. But yesterday you were asking for alternate solutions.
    <<< I did consider creating different folders for different users,
    but the issue with that approach, is that there may be many users
    with possible new users registering all the time. I will then have to
    add individually, each folder to be indexed, for each new user, in the Admin backend,
    which isnt feasible >>>

    Yes, there will be some other solutions. For example you could use the AND function of the search algorithm to combine the query term with the user name/id. But this approach would require modified scripts, as you would need to reject single word queries, and also the user id (folder name) needs to be added to any of the HTML head meta tags. Needs to be added to each page to be indexed. Also defining the user name would have to be generated automatically by random creating the 'id' for each user.
    Tec

  • 11/13/13--20:33: Re: How can I search for particular user documents
  • Thanks for taking the time to respond.
    I am going through the sphider source code if I can
    do this myself, if not I am planning to engage and pay someone to do
    this for me by posting in the Custom Mods section of this forum.
    Thanks again for the detailed answer.
    Thanks,

  • 11/15/13--05:47: Re: Error version number
  • 11/15/13--05:52: Re: Search this forum.
  • Tec Wrote:
    -------------------------------------------------------
    > Looks like an invalid script implementation into
    > an exxisting HTML layout. On the other hand, thee
    > original Sphider was developed about 7 years ago
    > by Ando Saabas. As he does not support the search
    > engine any longer, your issue might also be a
    > problem of deprecated PHP functions.
    >
    > Tec
    And there are alternative search engines?

  • 11/20/13--09:22: Indexing performance
  • Can anybody explain what factors have the biggest effect on Sphider's indexing speed? (Server memory, processor speed, available bandwidth?)
    I need to fully index about 500 websites, each with several hundred/thousands of pages, so I'd like to understand how to get the best performance from Sphider.
    I've tried searching for discussions about Sphider performance tips, but I cannot find any.

  • 11/20/13--12:33: Re: Indexing performance
  • Performance is mostly defined by the size of the database your indexation will create. As larger your db, as slower the index procedure will become. In other words: For the first x pages, the indexation runs quite fast. For the next x pages, the index algorithm again needs to verify, whether the words (text) found in current page to be indexed, are already known as stored keywords. For the next x pages, the indexation again becomes slower, as meanwhile the db content (count of keywords) is increased. And so on. As the Sphider is based on PHP scripts and a MySQL database, your may influence the indexation speed by adding additional server capability. But at least the indexer will become slower and slower, while the database size grows continuously . . .
    The same situation for Sphider-plus. But using this search engine, you may distribute the contents to be indexed on up to 5 databases. Additionally each of these 5 database allows you to define an unlimited count of table sets. Thus you may divide all your URLs into suitable (fast) subsets.
    Your available Internet bandwidth may assist you. But if the server, delivering the contents to be indexed, is limited in its bandwidth, there is no advantage for you.
    Tec

  • 11/22/13--17:46: Why are threads being deleted ?
  • Would a moderator chime in and explain why Whole Threads are being deleted ?
    example here i made a post with a link to another section of the forum, and it was there for quite some time.
    cladiron Wrote:
    -------------------------------------------------------
    > In case anyone is still watching this thread.
    > I have some what of an edit coming soon that does
    > multiple indexes.
    > Read more here.
    > [www.sphider.eu]
    Here is the thread to the post:
    http://www.sphider.eu/forum/read.php?3,7384,7384#msg-7384

  • 11/23/13--04:59: Re: Why are threads being deleted ?
  • The same to me. The search form of this forum counts 806 postings for 'Tec'. Never the less, hundreds of them meanwhile disappeared. Not taking into consideration all posting in the former Sphider forum, which was completely removed at the end of 2008.
    Tec

  • 11/23/13--06:50: Howto: Index using a script (Theme MOD and Mass-Indexing)
  • This is a repost of a deleted thread. I'm unsure if the admin here will delete this again, so it may not stay here long.
    I have posted my version edits here at http://sourceforge.net/projects/sphidercomunity/ for downloading.
    Any issues with the Sphider-CV should be posted in this thread.
    In this version you will have to manually add your links to the scripts.
    I am working on a way to automate this process. This has to be made so the links are viewable before pressing the button to index them. Just in case you need to remove sites that you do not want in your search.
    Excellent way to populate your search engine with little effort.
    Using scripts, you shouldn't have to worry about your site timing out when running the indexer.
    You will not be able to view each line yet, but that is a work in progress among a few other ideas.
    You will be able to view the status area in the admin section and watch the links and keywords increase.
    I'm not sure how many people may know about this or have give it much thought, but i found it to be a life saver.
    Due to how long sites can take to index, i found i lost alot of time when the indexer would finish while i was asleep.
    So i would miss hours that it could have been running.
    This little script can keep it indexing for days, depending how many you add to the script.
    In this little TUT i will explain how to create a script to place all your website links in for Indexing.
    Create an .sh file called what ever you like. I will call mine 'run_indexer_depth.sh'
    Now place the code below inside it, replacing the URL's and indexing depth.
    You can place as many web-URLs as you want in the file.
    (The # sign is a comment, and means that line will be skipped.)
    Even tho the line is not processed, it does still show in the console if viewing it. If you rather not have the messages show in the console, you CAN remove the lines that start with a # sign.
    Place your newly created run_indexer_depth.sh file in the same folder as spider.php
    Chown the .sh file to 755 (i use FTP to change the permissions)
    Now in SSH navigate to the .sh file and exec it.
    Use screen so you can close out SSH when you want.
    With screen Example:
    To close out the screen without closing out the indexer.
    While viewing the scan of the indexer press these:
    Without using screen Example:
    (if you start it this way, you must keep the SSH window open that is running the indexer. If you close it, the indexer WILL stop)
    This can also be used to Reindex your sites.
    Example below.
    (Must be ROOT or a SUDO-user)
    To install screen:
    CENTOS:
    UBUNTU:
    CRON
    Now this file can be setup to run as a Cron.
    Things to consider when setting up the Cron
    How large are the sites ?
    Depth your going to index.
    How many links you add to the .sh script.
    Servers stats with 8 groups running.
    This is the average or less, but i seen 2 spikes on the CPU that got over 5%.
    1 was at 25%, the other was at 10%
    Here is an Archived link of the orginal thread:
    [web.archive.org]

  • 12/03/13--00:50: Unknown error
  • Trying to run Sphider from the command line and am getting the following type error(s). I am assuming the 'Connection Refused' bit is the main issue. I can login to the MySQL database without a problem.
    After it is run there is a 'Site URL' of
    /.
    added to the Admin. Following that link brings me to the home page of my web site.
    php: /usr/local/lib/libpng12.so.0: no version information available (required by php)
    Set-Cookie: PHPSESSID=a06d8a46086900006c0d2c7dbe5c3fd9; path=/
    Expires: Thu, 19 Nov 1981 08:52:00 GMT
    Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
    Pragma: no-cache
    Content-type: text/html
    Connection refused<html><head><LINK REL=STYLESHEET HREF='admin.css' TYPE='text/css'></head>
    <body>[Back to <a href='admin.php'>admin</a>]<p><font size='+1'>Spidering <b>/</b></font></p>
    <b></b> at 09:40:51<br/><br>Completed at 09:40:51.
    <br></body></html>-bash-3.2$
    To confirm I am running the command correct, I am logging into the server using SSH, navigating to the directory where spider.php resides and entering:
    -bash-3.2$ php spider.php -u [www.site_to_be_indexed.com]
    Thank you in advance :)
    The fray scars and stories album.

  • 12/05/13--17:09: JOYAL machines est de s'adapter au modèle de développement de l'entreprise moderne
  • Ici, nous allons analyser brièvement les principaux avantages de concasseur à cône rapport avec concasseur à mâchoires .
    1. Opération Stable , base de faible poids et les vibrations de la lumière. Le poids de base de concasseur à cône est généralement 2-3 fois plus que de l'équipement de machines . Alors que le poids de base de concasseur à mâchoires était de 5 à 10 fois de la machine elle-même .
    2.Cone concasseur est plus facile de commencer par rapport à la mâchoire de granit concasseur . Avant le début d'un volant de rotation lourde , concasseur à mâchoires a besoin d'aide supplémentaire .
    3 . Grand concasseur à cône peut être directement alimenté par le silo à minerai et du minerai brut . Il ' s pas besoin d' alimentation supplémentaire. Concasseur à mâchoires n'est pas emballé à la mine , et même demander à la mienne . Par conséquent , un bac de séparer le minerai est nécessaire , en particulier lorsque le bloc de minerai est plus grand que 400 mm , vous devez installer cher lourde plaque d'alimentation .
    Concasseur à cône est largement utilisé dans de nombreuses industries et présente les caractéristiques d' une grande efficacité. Selon les différents requirments de broyage , il peut être divisé en trois types : concassage grossier , concassage secondaire et tertiaire broyage . Sur la base de la situation actuelle en Chine , le concassage secondaire utilisent habituellement concasseur à cône standard et concassage tertiaire utilisent généralement de type court de la tête concasseur à cône . Cependant, le matériel de concassage grossier est concasseur à cône ou concasseur à mâchoires .
    Notre ZYS concasseur à cône a une variété de types et de caractéristiques différentes. PYB printemps concasseur à cône est nos produits brevetés et intègre les avantages des autres types de concasseur à cône . Il s'agit d'une grande efficacité , super fine écrasement , économie d'énergie et des équipements de concassage mobile de nouvelle de type avec le principe et la structure uniques . Comme l'une des infrastructures les plus importantes qui appuient la croissance future , l'exploitation minière industrie des machines a pris de plus en plus d'attention . Pour lancer de nouveaux produits avec les caractéristiques de protection de l'environnement et la conservation de l'énergie est de plus en plus la tâche urgente dans l'industrie de concasseur . Concasseur à cône produite par JOYAL est un équipement de concassage représentant . Il est équipé de la meilleure qualité et une grande efficacité , dans une certaine mesure , il stimule l'ensemble du développement de l'industrie du minerai de cuivre concasseur .

  • 12/05/13--18:35: parajumpers damen jacken sale
  • <P>Diese Internetseite stellt allgemeine Informationen zur Verfügung und dient [www.pjs-parajumpers-jacken.com] lediglich Aufklärungs-Zwecken. Wenn Sie irgendein Interesse auf Ihre eigene Gesundheit oder die Gesundheit Ihres Kindes haben, sollten [www.pjs-parajumpers-jacken.com] immer mit einem Arzt oder anderen Angehörigen des Gesundheitswesens aufsuchen. Bitte überprüfen Sie die Nutzungsbedingungen, bevor Sie diese Website.r <P>Sie müssen möglicherweise nicht stecken Kauf Krankenversicherung, die nicht gehen wird, um Ihnen die Aufmerksamkeit, die Sie wollen, zu helfen. Krankenversicherung muss eine der größten wirtschaftlichen Priorität auf Ihrem eigenen Angebot sein. [www.pjs-parajumpers-jacken.com] 1997-2013 Alle Rechte vorbehalten. Diese Internetseite stellt allgemeine Informationen zur Verfügung und dient lediglich Aufklärungs-Zwecken. Wenn [www.pjs-parajumpers-jacken.com] irgendein Interesse auf Ihre eigene Gesundheit oder die Gesundheit Ihres Kindes haben, sollten Sie immer mit einem Arzt oder anderen Angehörigen des Gesundheitswesens aufsuchen. Bitte überprüfen Sie die Nutzungsbedingungen, bevor Sie diese Website.r <P>Von Bergen bis Umhänge, ruft der wärmegedämmten St Moritz Wasser-und winddicht Ski cape von der sportlichen Erfolge von verschwitzten Bettys Schnee Bereich den Glamour der 60er Jahre mit verstellbarem Gurt und fertig mit Pelzverbrämungen und abnehmbarem Pelzkragen. Anzeigen Ihre Lizenz auf Nervenkitzel, wird der Ski-Overall Cresta bringen auch das Bond-Girl in Ihnen schmeichelhaft schwarze Kontur mit Fleece-Futter geschnitten und garniert mit einem hohen Hals Fell trimmen. 'Bonding' über Vintage Reiz, werden diese aus, um Perfektion in wahre 007 Style mit der großen Auswahl an Pelz Kosaken und Ushanka Hüte im Angebot [www.pjs-parajumpers-jacken.com] oder John Richmond Silberfuchs Ohrenschützer beendet.r <P>Dies kann hilfreich sein. Dennoch ist es sehr wichtig, sie aufzunehmen. [www.pjs-parajumpers-jacken.com] Sie werden Ihnen helfen, ein paar Möglichkeiten für Sie persönlich vor, Sie erhalten eine endgültige Auswahl [www.pjs-parajumpers-jacken.com] sollte. Wenn Sie wirklich wollen, zu verstehen, was Angst, als zu versuchen, um zu leben Jakke Parajumpers ein paar Tage ohne Krankenversicherung ist. Sie werden schnell besorgt über jeden kleinen Husten oder Schnupfen. Brick-and-Mörtel Broker und Versicherungen kosten Versicherungen Geld für ihre Miete, Gewinn, Nutzen für die Gesundheit, und so weiter zu zahlen.r <P>Sie werden helfen, [www.pjs-parajumpers-jacken.com] paar Möglichkeiten für Sie sammeln, kurz bevor Sie eine endgültige Entscheidung zu treffen haben. Suchen Sie einfach eine in der anderen, und auf keinen Fall kaufen die beiden. Es gibt viele [www.pjs-parajumpers-jacken.com] Websites, die Ihnen helfen, zu beginnen, um die Preise von mehreren verschiedenen Agenturen auf einmal sehen.r <P>Ein Selbstbehalt ist auf [www.pjs-parajumpers-jacken.com] Fall die Kosten werden von Jakke Parajumpers [www.pjs-parajumpers-jacken.com] Brieftasche während Unterstützung zu zahlen. Diese Preisgestaltung ist weit über die Kosten Ihrer wirklichen Versicherungsprämien. Noch im wesentlichen, warten Sie nicht, bis es zu weit weg ist, um Versicherung überhaupt kaufen, und erwerben verpflichtet, Zerkleinern Gesundheit Aufwendungen.

  • 10/01/13--14:00: Re: Auto Index (IIS)?
  • Did you fix this? ever try creating a shedule task on the windows OS end?

  • 10/02/13--22:57: Re: SEO Friendly Dynamic Page Titles
  • SEO plays a major role in promotion of a site and hence the business. This is very important for a business to excel. This post is very very essential and beneficial to read for the ones who are working in this field. It has helped me a lot. Thank you for this post. Thank you very much. :)
    Visit:
    creative web design search engine marketing company

  • 12/11/13--21:16: How to implement the relevant search results related keywords?
  • How to implement the relevant search results related keywords, no site index list page. Indexing time need to retrieve the list of pages, but I hope the search results page no article list page

  • 12/13/13--22:29: Re: Howto: Index using a script (Theme MOD and Mass-Indexing)
  • Vox pathfinder 15 watt review. This is really very important script to index the pages in search engine. this script is really helpful my site.
    picnic spots near delhi botanix nature resort

Sphider Pro is a PHP search engine based around the idea of the original Sphider by Ando Saabas.
We have taken the original scripts and rebuilt then from the ground up to make Sphider Pro a light weight dynamic, simple install package to run a powerfull PHP search engine on your website. Sphider Pro has its own bot to index internet content and images.
We have worked with a UK Web Hosting Company to make sure that this version of Sphider can and will work on shared web hosting packages. They will even install it for you! Webreger.com
Support is avaliable via our support 24/7. Help is always on hand.
Sphider Pro comes with very little CSS styling and is ready for your to implement your style around our code scripts.
This version has been tested with over two million keywords, 900,000 links, 29,540 domains & 37,000 images My Safe Search
Sphider Pro comes with full support via our support desk, we have dedicated members who are on hand to help with any part of Sphider Pro.
For a full list of features, updates and installation instructions please visit our support knowledge base.
Sphider Pro comes with pre installed themes. These can be set in the admin control settings panel. Custom themes can be created and uploaded so these can also be selectable. For help with Sphider Pro custom themes please see Webreger.com how can help build themes.
Sphider Pro can take a screen shot of a website while indexing and these can then be displayed in the search results page. This setting can be turned off in admin control settings panel.
Sphider pro allows you to activate this feature in the settings control panel. users will then have the option to report a url. this is shown next to the url in the search results page.
This setting if activated will allow users to suggest a website to be included in the Sphider Pro indexing list.
Sphider Pro comes with full support to index and search using multiple languages, such as Arabic, search terms can be entered in Arabic and results will be displayed in the websites native language.
Sphider Pro allows indexing of all pdf and doc files. Simply activate this from within admin control settings and enter the path to your converter. (These are installed on all good web servers).
Periodical re-index all sites via cron jobs, Once set in edit sites and cron job is enabled on your server all sites your have selected will be indexed / re-indexed in accordance with the cron job time settings.
To see what pages have been indexed just like Google you can use site:yourdomain.com this will display all the pages that have been indexed relating to that website. Try site:www.webreger.com in our demo.
Displays how long the search query took like Google. I.e. Displaying results 1 - 10 of 2378 matches (0.8 seconds).
Index and search for images, this feature allows for image search like Google. Images are indexed by src, width, height and alt tags.
Individual configuration and activation of databases for Admin and Search Users. This allows admin to have full database permissions but granting only basic permissions for the Search users. This is a security feature added in and recomended.
This featured has been impemented to the search box allowing auto suggestions like Google. Suggested querys are formated based on previous Search users terms, keywords tables & suggested phrases. This also prompts the related search function.
If a result returns no rows the system suggests spelling alternatives. These are populated from keywords and search query logs to enable a did you mean function. This function is turned on from the admin area.
Sitemaps are auto detected and links within the sitemaps will be indexed. This feature can be turned off within the admin area.
Robot text files are always respected. Sphider Pro will not follow or index and page url's that are disalowed by robot text files
Sphider Pro respects robot meta tags and will not index or not follow depending on data set within these tags.
Sphider Pro indexes data with meta tags. Title and description meta data will be used for searching and displaying in the search results page. Web pages that do not have meta tag description then limited page content will displayed in the search results.
Where a page returns a 404 error these pages are logged and removed by the clean tables function.
This function selects domains at random to be re-indexed. After re-index has completed the function removes blocked web url's, blocked images, cleans temp tables and removes 404 error pages.
This function selects domains at random that have not yet been index by Sphider Pro an indexes them. After indexing has completed the function removes blocked web url's, blocked images, cleans temp tables and removes 404 error pages.
With this feature you can block a url from being index if it has passed the disallow feature but you don't want that page indexed. This feature is available via the sphider Pro indexing interface.
If an image has been indexed but you do not wish this to be part of the data base you can ban this image. This feature is available via the sphider Pro indexing interface.
Individual (site specific). Removes the site, links associated with the site and keywords ready to be re-indexed.
Various modes for sorting search results such as weight of meta keyword, meta description, page title and more. These are selectable via the admin interface.
Search log, Most popular text links, Most popular media links, User IP, Last queried, Top keywords, etc.
Page must contain any / all of the words in whitelist. Admin selectable is also a blacklist, holding words to prevent indexing of pages containing these forbidden words.
Sphider Pro respects attribute in <a href.> tags, so for example the link test.html in <a href='test.html'> is ignored.
This feature once selected in admin area will allow advanced searching features such as Phrase search and category search if categories have been activated.
If category search is selected users can search via categories. Admin can turn this feature on or of in. Categories must be created and sites diffined by category for this feature to work. Also Advanced search must be turned on.