ListPages is very slow to process

Bugs

---

---

Tags

backlinks block bug button code css disabled div edit-sections file forum hidden include listpages lock manager recognizing rename save slow tag timeout upload

Not completely fixed. Still happens some of the times.

Description

The ListPages module, under certain conditions, is extremely slow to load/process (sometimes taking a couple of minutes). This appears to be a processing lag on Wikidot's end rather than a connection speed issue. The lag occurs even on page lists that consist only of text and would normally take less than a second to load. The time for a list pages to process appears to increase dramatically when ListPages searches a category with a lot of pages.

Once a page with ListPages loads, its contents appear to stay "cached" by Wikidot. Therefore, it only affects the first visitor to the page, as well as visitors that go to the page after it leaves the Wikidot "cache".

Here is a demonstration of what happens:

• Person A visits a ListPages page, waits 1-2 minutes for it to process and load, then leaves the site.
• About 15 minutes later, person B goes to the same page. Person B will see the page instantly.

As seen in this example, the initial visit triggers Wikidot to process the ListPages and keep it cached. All visitors coming soon after person A will be able to view the page instantly. I have tested this using multiple computers to prove that it is not a matter of data being cached locally.

This is a problem because:

• The first person will likely not wait 2 minutes and will quickly leave, so the page will never get cached by Wikidot.
• The pages don't stay cached forever… If person B (in above example) went back to the page 4 hours later, they would have to endure the massive wait while Wikidot reprocesses it (assuming no one else visited it right before them).

How to Reproduce

To expose this problem, it's best if you have a lot of pages. Here's an example from my wiki, which currently has 200+ pages in the category ListPages is searching.

As described above, once a page is visited it will load fast for all subsequent visitors (only for a while). I included several copies of my page so you can find one that loads slowly.

http://scmapdb.wikidot.com/tag:boms-listpagetest
http://scmapdb.wikidot.com/tag:boms-listpagetest2
http://scmapdb.wikidot.com/tag:boms-listpagetest3
http://scmapdb.wikidot.com/tag:boms-listpagetest4
http://scmapdb.wikidot.com/tag:boms-listpagetest5
http://scmapdb.wikidot.com/tag:boms-listpagetest6
http://scmapdb.wikidot.com/tag:boms-listpagetest7
http://scmapdb.wikidot.com/tag:boms-listpagetest8

See this thread in which other users confirm the bug.

Browsers

Experienced in all browsers.

Workarounds

To keep your ListPages loading fast for everyone, you can:

• Manually visit the page constantly throughout the day, so Wikidot keeps caching it.
• Setup some kind of program or bot to automatically visit your pages that have a ListPages module.

Note: The wait time is drastically reduced if you paginate your content by using a low "perPage" value.

Contact

mmclean89

Rate this Bug

Rate the urgency of this bug. If you think it is more urgent and important than it's current rating suggests, rate it up.

Comments

probably related to tags

albertoxx 1215216747|%e %b %Y, %H:%M %Z|agohover

I suffer from this issue too if I specify both category and tag in ListPages.
For example, the following returns a reasonably prompt output

[[module ListPages category="*"]]

whereas the following results in a latency as described in this bug report

[[module ListPages category="*" tag="some_tag"]]

Re: probably related to tags

David Marseilles 1215286071|%e %b %Y, %H:%M %Z|agohover

You might be on to something. I've never seen my main page lag (has no tag limts), but my secondary blog does have tag limits, and it does hang on me sometimes. I used to think it was just the accompanying RSS feeds, but I recently separated them and the blog can still hang.

Gabrys 1215342853|%e %b %Y, %H:%M %Z|agohover

Hi, we'll take a look at this soon.

---

Piotr Gabryjeluk
visit my Dev blog

Re: ListPages is very slow to process

David Marseilles 1215370457|%e %b %Y, %H:%M %Z|agohover

Thank you =)

FIXED

Gabrys 1215417867|%e %b %Y, %H:%M %Z|agohover

Hi all,

we've fixed this issue.

Please verify this.

---

Piotr Gabryjeluk
visit my Dev blog

Re: FIXED

David Marseilles 1215418419|%e %b %Y, %H:%M %Z|agohover

Off-hand, my tag limited page did just load with no lag.

Was the problem related to tags?

Re: FIXED

mmclean89 1215459547|%e %b %Y, %H:%M %Z|agohover

All my pages with ListPages now load pretty much instantly. Thanks so much!

Re: FIXED

albertoxx 1215545099|%e %b %Y, %H:%M %Z|agohover

I confirm it.
I don't observe the latency I experienced before.

Re: FIXED

Sulfrum 1215933042|%e %b %Y, %H:%M %Z|agohover

Yes, it works!

Ent

---

You'll pay for all your sins. If you already paid, please disregard this message.

Re: FIXED

Sulfrum 1215977084|%e %b %Y, %H:%M %Z|agohover

Today when I was using the module I found a bug (can someone confirm that it's really a bug?).
At the bottom of the message there is the link and the source of the page.

Description of the bug:
I use the ListPagesModule with collapsible formatting. If there are some other collapsible before the module, when I click on the collapsible in the module it doesn't open; It open the first or the second. There are no way to open the first and second collapsible in the module.

Sulfrum

This is the source of this page:

+ Collapsible 1
[[collapsible show="+ open" hide="- close"]]
UNO
[[/collapsible]]
+ Collapsible 2
[[collapsible show="+ open" hide="- close"]]
DUE
[[/collapsible]]
+ ListPagesModule with collapsible formatting
[[module ListPages category="obj" tags="ar_accademia-magia -old_obj" order="titleAsc" rssShow="no" perPage="10" separate="yes"]]
[[collapsible show="+ apri" hide="- nascondi"]]
%%content%%
[[/collapsible]]
[[/module]]

---

You'll pay for all your sins. If you already paid, please disregard this message.

Re: FIXED

David Marseilles 1215981457|%e %b %Y, %H:%M %Z|agohover

+ ListPagesModule with collapsible formatting
[[module ListPages category="obj" tags="ar_accademia-magia -old_obj" order="titleAsc" rssShow="no" perPage="10" separate="yes"]]
%%linked_title%%
[[/module]]

Can you place this on a page, or below all that on the same page? I'd be interested to see what is on those first two pages.

Also, this might not help, but I've experienced some problems with collapsibles being too close to other content before, try putting an empty line (return) before and after each collapsible module.

Example:

+ Collapsible 1

[[collapsible show="+ open" hide="- close"]]
UNO
[[/collapsible]]

+ Collapsible 2

[[collapsible show="+ open" hide="- close"]]
DUE
[[/collapsible]]

+ ListPagesModule with collapsible formatting
[[module ListPages category="obj" tags="ar_accademia-magia -old_obj" order="titleAsc" rssShow="no" perPage="10" separate="yes"]]

[[collapsible show="+ apri" hide="- nascondi"]]
%%content%%
[[/collapsible]]

[[/module]]

last edited on 1215981494|%e %b %Y, %H:%M %Z|agohover by David Marseilles + show more

Re: FIXED

Sulfrum 1216066105|%e %b %Y, %H:%M %Z|agohover

I've done the page Fortunzfavor asked me.
The insertion of empty line doesn't change nothing, the bug remain.
Thanks for the feedback, I'll try some other experiments as soon as possible.

here is the page.

Sulfrum

---

You'll pay for all your sins. If you already paid, please disregard this message.

Re: FIXED

vaguito 1216076110|%e %b %Y, %H:%M %Z|agohover

Every collapsible block is compiled into two divs whose visibility is exchanged when toggled by the user.

These divs are identified as «collapsible-block-N-folded» and «collapsible-block-N-unfolded», where N is, I believe, an unique block number.

The problem is this numbering is restarted when the module is evaluated. Therefore, the first pair of pages listed will have the same id as those collapsibles outside the module.

I think there's not much you can do by tweaking the wiki page source.

This behaveour¹ also arises when collapsibles are both on the sidebar and the main content. This is actually different, a collapsible on the sidebar may control one in the main content but cannot work alone.

Footnotes

1. Note: This word is possibly badly spelled.

last edited on 1216319514|%e %b %Y, %H:%M %Z|agohover by vaguito + show more

Re: FIXED

Gabrys 1216151392|%e %b %Y, %H:%M %Z|agohover

Just delete separate=yes.

---

Piotr Gabryjeluk
visit my Dev blog

Re: FIXED

vaguito 1216173200|%e %b %Y, %H:%M %Z|agohover

It seems that is not helping. Have a look at this test page: http://desertline.wikidot.com/sandbox:test-collapsible

Re: FIXED

vaguito 1216485862|%e %b %Y, %H:%M %Z|agohover

Here is another test page, http://desertline.wikidot.com/sandbox:test-collapsible-2 , headers were reordered just in case.
It looks like separate does not make any difference.

I think this, starting from post #post-219591 by Sulfrum, should be moved to another thread with a more suitable title…like "collapsible blocks' id clashing under certain conditions".

last edited on 1216497442|%e %b %Y, %H:%M %Z|agohover by vaguito + show more

Re: FIXED

mmclean89 1219984804|%e %b %Y, %H:%M %Z|agohover

while it's much faster, listpages is still not instantaneous when processing a large amount of stuff. Why does it generate the list when someone visits it? This behavior will always cause a delay. Can't it be pre-generated before hand so it loads up instantly when requested?

3-4 seconds waiting is still enough make someone give up and leave.

edit: Now i'm getting a couple of minutes delay on some pages.

last edited on 1220034090|%e %b %Y, %H:%M %Z|agohover by mmclean89 + show more