| Page 1 of 2 |
headshot
VIP Member
Posts: 34961
Location: UK
|
 Posted: Sun, 13th Sep 2020 17:49 Post subject: Site Search Broken? |
|
 |
Is the site search broken?
Example 1: I search for the thread 'Graven' which I know exists in the PC Games Arena section. Search tells me 'No topics or posts met your search criteria'.
I then manually search the first few pages of the PC Games Arena and find it.
https://www.nfohump.com/forum/viewtopic.php?t=115539
Example 2: I search for the thread 'Black Myth' again which I know exists in the PC Games Arena section. Search doesn't list the thread in the results.
I then manually search the first few pages of the PC Games Arena and find it.
https://www.nfohump.com/forum/viewtopic.php?t=115491
So is search broken, has something changed or am I doing something?
May the NFOrce be with you always.
|
|
| Back to top |
|
|
Morphineus
VIP Member
Posts: 24883
Location: Sweden
|
| Posted: Sun, 13th Sep 2020 18:05 Post subject: |
|
|
Nah its not you, search has always been kind of broken. A bit more the past months. Have seen others complain about it.
|
|
| Back to top |
|
|
headshot
VIP Member
Posts: 34961
Location: UK
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Sun, 13th Sep 2020 18:36 Post subject: |
|
|
When has the search not been broken?
I don't understand why it's so hard to just have an index of titles and messages and just run a LIKE query with COLLATE NOCASE?
|
|
| Back to top |
|
|
ixigia
[Moderator] Consigliere
Posts: 65006
Location: Italy
|
| Posted: Sun, 13th Sep 2020 23:05 Post subject: |
|
|
Yeah, I'm a complete Jon Snow when it comes to this sort of stuff but I imagine it's something rather straightforward for those who are blessed with database omnipotence. *does the naked dance in order to summon Mrt*
Our black hole-powered search got worse when the Bad Query message also started to randomly appear, now it often just "forgets" about threads even though they are still accessible. At least it doesn't delete them in the process, it's something I guess  
|
|
| Back to top |
|
|
[mrt]
[Admin] Code Monkey
Posts: 1338
|
| Posted: Wed, 16th Sep 2020 20:38 Post subject: |
|
|
| LeoNatan wrote: | When has the search not been broken?
I don't understand why it's so hard to just have an index of titles and messages and just run a LIKE query with COLLATE NOCASE? |
Rebuttle: its is the reason the forum was slow and bogged-down as the archive of posts on this forum is massive over a million posts and it was just too large for that. Sphinx is used as the search indexer and it should work fine. The indexer updates evey 5 minutes so at worst case the search results are 5 minutes old.
@headshot By defailt the option "Search topic title only" is selected if you want better and broader results try using "Search topic title and message text" to search both titles and texts.
teey
|
|
| Back to top |
|
|
tonizito
Posts: 51254
Location: Portugal, the shithole of Europe.
|
| Posted: Wed, 16th Sep 2020 20:44 Post subject: |
|
|
And when all fails if you use google search it links you to the threads that "fall off" 
| boundle (thoughts on cracking AITD) wrote: | | i guess thouth if without a legit key the installation was rolling back we are all fucking then |
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Wed, 16th Sep 2020 21:19 Post subject: |
|
|
| [mrt] wrote: | | LeoNatan wrote: | When has the search not been broken?
I don't understand why it's so hard to just have an index of titles and messages and just run a LIKE query with COLLATE NOCASE? |
Rebuttle: its is the reason the forum was slow and bogged-down as the archive of posts on this forum is massive over a million posts and it was just too large for that. Sphinx is used as the search indexer and it should work fine. The indexer updates evey 5 minutes so at worst case the search results are 5 minutes old.
@headshot By defailt the option "Search topic title only" is selected if you want better and broader results try using "Search topic title and message text" to search both titles and texts. |
It doesn’t seem to discover all posts. Often threads are just missing from the results. If it was just the delay, I don’t think people would notice or complain.
Is the search slowness due to the old db software used? |
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
|
| Back to top |
|
|
M4trix
Posts: 9140
Location: Croatia, Adriatic coast (I can see ixi from here)
|
|
| Back to top |
|
|
|
|
| Posted: Wed, 16th Sep 2020 22:59 Post subject: |
|
|
By default, full-text search solutions make the minimum keyword length to 4 chars. With 3 chars or less you definitely need to implement stopwords tables.
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Wed, 16th Sep 2020 23:12 Post subject: |
|
|
But it finds other threads with “AMD” in the title. The length isn’t the problem. Unclear why, but some threads just aren’t indexed.
|
|
| Back to top |
|
|
ixigia
[Moderator] Consigliere
Posts: 65006
Location: Italy
|
| Posted: Wed, 16th Sep 2020 23:35 Post subject: |
|
|
I've tried the search function for the most recent thread on the PC Arena out of curiosity (Demon's Souls), and it doesn't appear either no matter the combo or specific search method used. Full name, Demon, Demon's, Souls, the thread is always hidden whereas the console one is there. There are quite a few other examples as well with relatively long words that don't show up
|
|
| Back to top |
|
|
headshot
VIP Member
Posts: 34961
Location: UK
|
|
| Back to top |
|
|
|
|
|
| Back to top |
|
|
|
|
| Posted: Thu, 17th Sep 2020 11:31 Post subject: |
|
|
Last edited by Interinactive on Mon, 4th Oct 2021 08:50; edited 3 times in total
|
|
| Back to top |
|
|
|
|
| Posted: Thu, 17th Sep 2020 13:05 Post subject: |
|
|
If I search using the term "asshole", the usual suspects are found. So seems to be working fine. Just an FYI.
|
|
| Back to top |
|
|
Frant
King's Bounty
Posts: 24555
Location: Your Mom
|
| Posted: Thu, 17th Sep 2020 15:32 Post subject: |
|
|
| LeoNatan wrote: | | But it finds other threads with “AMD” in the title. The length isn’t the problem. Unclear why, but some threads just aren’t indexed. |
Perhaps it's simply a query bug where it doesn't go through all the iterations/keys from the database due to some formatting issue or a missing variable/index identifier. It could be anything from a formatting error to a more complex issue with the php-query script, I dunno.
Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn!
"The sky was the color of a TV tuned to a dead station" - Neuromancer
|
|
| Back to top |
|
|
Nalo
nothing
Posts: 13491
|
| Posted: Thu, 17th Sep 2020 22:03 Post subject: |
|
|
Last edited by Nalo on Wed, 3rd Jul 2024 05:43; edited 3 times in total
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Thu, 17th Sep 2020 22:15 Post subject: |
|
|
| Frant wrote: | | LeoNatan wrote: | | But it finds other threads with “AMD” in the title. The length isn’t the problem. Unclear why, but some threads just aren’t indexed. |
Perhaps it's simply a query bug where it doesn't go through all the iterations/keys from the database due to some formatting issue or a missing variable/index identifier. It could be anything from a formatting error to a more complex issue with the php-query script, I dunno. |
That doesn’t make sense given how there are some results. If there is a bug, the bug is at indexing time, not at query time. |
|
| Back to top |
|
|
Frant
King's Bounty
Posts: 24555
Location: Your Mom
|
| Posted: Fri, 18th Sep 2020 00:27 Post subject: |
|
|
I don't know how the database is set up, nor do I know how the indexer is working. It's somewhere between the query and the database responding to a command (indexer's function to poll data from the database/dataset).
I haven't coded with/against SQL since ~2005 or so except for a few tests on remote servers to see if key generation for each object etc. worked. I keep saying to myself that I need to get back to coding but then I also have to get back to playing and writing music as well as a ton of other stuff so.... and I'd really like to get into retro coding on the Amiga (68K assembly) as well as play around with 65xx assembly on the C64 and see if I can figure out how to write a raster interrupt and put a proper sprite scroll up as well as moving raster bars etc., things I haven't done since pre-1990 (except dabbling a couple of times in emulators). I also want to learn Z80 assembly language (which is using pretty much the same instructions as the old Intel CPU instruction set (JP, POP etc. or something).
Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn!
"The sky was the color of a TV tuned to a dead station" - Neuromancer
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Fri, 18th Sep 2020 00:30 Post subject: |
|
|
It’s a third party indexer, which runs independently of the forum software. It doesn’t runs a real-time query if the main forum database on query. As mrt said, there is a 5 minute delay, because it crawls over the database and indexes new posts.
|
|
| Back to top |
|
|
[mrt]
[Admin] Code Monkey
Posts: 1338
|
| Posted: Sat, 26th Sep 2020 15:42 Post subject: |
|
|
Okay that should do it. Threads were missing because the sphinx configuration got out of sync from the main db and the indexing stopped. Its all been rebuilt now and the missing threads reappeared again.
indexing index 'nforceforum'...
collected 3217763 docs, 1144.7 MB
sorted 196.4 Mhits, 100.0% done
total 3217763 docs, 1144678279 bytes
total 130.757 sec, 8754187 bytes/sec, 24608.57 docs/sec
mysql> SELECT * FROM `nforceforum` WHERE id = 3306069;
Empty set (0.00 sec)
mysql> SELECT * FROM `nforceforum` WHERE id = 3306069;
+---------+----------+----------+-----------+------------+--------------+------------+---------------+
| id | topic_id | forum_id | poster_id | post_time | topic_poster | topic_time | topic_replies |
+---------+----------+----------+-----------+------------+--------------+------------+---------------+
| 3306069 | 115532 | 5 | 5123 | 1599230260 | 5123 | 1599230260 | 117 |
+---------+----------+----------+-----------+------------+--------------+------------+---------------+
1 row in set (0.00 sec)
Keep on eye on it all peeps and let me know how it goes.
teey
|
|
| Back to top |
|
|
Frant
King's Bounty
Posts: 24555
Location: Your Mom
|
|
| Back to top |
|
|
tonizito
Posts: 51254
Location: Portugal, the shithole of Europe.
|
| Posted: Sat, 26th Sep 2020 16:28 Post subject: |
|
|
Very nice, good work 
| boundle (thoughts on cracking AITD) wrote: | | i guess thouth if without a legit key the installation was rolling back we are all fucking then |
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Sat, 26th Sep 2020 18:19 Post subject: |
|
|
|
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Sat, 26th Sep 2020 18:20 Post subject: |
|
|
Still broken unfortunately:
This thread still doesn't appear.
|
|
| Back to top |
|
|
[mrt]
[Admin] Code Monkey
Posts: 1338
|
| Posted: Sat, 26th Sep 2020 19:21 Post subject: |
|
|
It does. I left the search hit limit too low, it shows up again now. It may need some more tweaking so let me know if anything goes missing that it shouldn't.
teey
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Sat, 26th Sep 2020 19:42 Post subject: |
|
|
|
|
|
| Back to top |
|
|
|
|
| Posted: Sat, 26th Sep 2020 20:11 Post subject: |
|
|
@mrt still getting Bad Query from time to time. Usually another search (exactly the same) works.
PC: Yes. Console: No.
|
|
| Back to top |
|
|
| Page 1 of 2 |
All times are GMT + 1 Hour |
