02:08:41 <erdgeist> Do I understand correctly that dependencies in nikola are only to detect if certain artefacts need to be rebuilt, but not to determine the order in which posts need to be rendered the first time? 13:40:24 <ralsina> erdgeist: yes, the order once all dependencies are sorted out is not known 14:24:00 <ChrisWarrick> erdgeist: however, a post could depend on another post 14:53:00 -travis-ci:#nikola- getnikola/nikola#4662 (master - 37ebddc : Chris Warrick): The build is still failing. 14:53:01 -travis-ci:#nikola- Change view: https://github.com/getnikola/nikola/compare/7363b868e51b...37ebddc5af0c 14:53:01 -travis-ci:#nikola- Build details: http://travis-ci.org/getnikola/nikola/builds/53243206 15:05:33 -travis-ci:#nikola- getnikola/nikola#4662 (master - 37ebddc : Chris Warrick): The build was fixed. 15:05:34 -travis-ci:#nikola- Change view: https://github.com/getnikola/nikola/compare/7363b868e51b...37ebddc5af0c 15:05:34 -travis-ci:#nikola- Build details: http://travis-ci.org/getnikola/nikola/builds/53243206 18:49:18 <erdgeist> ChrisWarrick: well, for all the index pages the dependencies must be re-evaluated each time a new post is added. It might change the years index and the tags index 18:49:44 <erdgeist> ChrisWarrick: how does doit figure out, that these pages need to be rebuilt? 18:49:58 <ChrisWarrick> erdgeist: index pages depend on post files/pages 18:50:24 <ChrisWarrick> erdgeist: the post files change (not sure if it’s posts/, cache/, or output/) and this triggers a rebuild 18:51:21 <ChrisWarrick> erdgeist: furthermore, the post creation task has the files as their targets, which is also responsible for dependency calculation 18:53:13 <erdgeist> ChrisWarrick: Okay, and pages are different? I only noticed dependecies added AFTER the pages have been rendered 18:53:50 <ChrisWarrick> erdgeist: dependencies are set when tasks are generated 18:55:22 <ChrisWarrick> erdgeist: how did you notice “dependencies added after pages rendered”? 18:55:26 <erdgeist> ChrisWarrick: Maybe I am not making myself clear enough ;) There seems to be a concept of index pages that need to be rendered last 18:55:49 <erdgeist> ChrisWarrick: the .dep-File is only generated after the html 18:56:06 <erdgeist> ChrisWarrick: and then re-read 18:56:13 <ChrisWarrick> erdgeist: yeah, but the .dep files are used only for dependencies from reStructuredText contents 18:56:17 <ChrisWarrick> erdgeist: index pages must be rendered after all cache/ files that exist in this index page 18:56:32 <erdgeist> ChrisWarrick: correct 18:56:58 <erdgeist> ChrisWarrick: And I need that for pages that contain post-list directives 18:57:13 <erdgeist> ChrisWarrick: because they obviously depend on other pages 18:57:26 <ChrisWarrick> erdgeist: they depend on other pages if and only if you use post.text in the template 18:57:43 <erdgeist> ChrisWarrick: nope 18:57:56 <erdgeist> ChrisWarrick: did you get my pastie? 18:58:06 <erdgeist> 22:41 < erdgeist> Now I already get a stack trace from doit when using a post-list-directive: http://pastie.org/10003313 18:58:52 <erdgeist> the template there does not use anything from the post 18:58:55 <ChrisWarrick> erdgeist: what is stories/music/index.rst? 18:59:42 <erdgeist> .. post-list:: :all: :tags: Liedermacher :id: singersongwriter 19:00:00 <ChrisWarrick> and what is your template? 19:00:19 <erdgeist> the default from base 19:02:11 <erdgeist> and the missing Fine.html would be one of the pages aggregated from this post-list 19:02:42 <erdgeist> maybe its a bug in doit, idk 19:03:25 <erdgeist> it fails, because it cant save the dependency's file stamp 19:03:47 <erdgeist> because dependency was calculated but not yet generated 19:05:18 <ChrisWarrick> erdgeist: paste output of nikola dumpdb 19:06:40 <erdgeist> DBM type is 'dbm' 19:06:41 <erdgeist> ERROR: ndbm does not support iteration of elements 19:07:52 <ChrisWarrick> sheesh 19:08:06 <erdgeist> kurwa 19:08:24 <ChrisWarrick> tell you what, I can’t help you 19:08:36 <ChrisWarrick> I can’t see ANY dependency creation in post_list 19:09:03 <erdgeist> :) 19:09:30 <erdgeist> I would just add a header to posts containing post-lists, so they are de-prioritized 19:10:00 <ChrisWarrick> I would tackle your fancy indexes differently 19:10:16 <ChrisWarrick> Just use regular indexes and stick some extra text on the first page 19:10:31 <erdgeist> let me show you 19:10:56 <erdgeist> ChrisWarrick: https://erdgeist.org/new/ 19:11:42 <erdgeist> ChrisWarrick: the list of projects and blog post teasers is generated, and it works reasonably well 19:12:07 <erdgeist> ChrisWarrick: I just need a way to have some pages treated like index pages and rendered last 19:13:01 <ChrisWarrick> You could try doing this by making sure post.fragment_deps() has all the posts displayed enlisted 19:14:38 <ChrisWarrick> this will require a few changes to the nikola source code, it’s not trivial though 19:15:21 <ChrisWarrick> you could modify post_list_directive so that it talks to record_dependencies and hope for the best 19:15:40 <ChrisWarrick> self.state.document.settings.record_dependencies.add(fpath) 19:17:24 <ChrisWarrick> where fpath is an individual file path, should be AT LEAST post.source_path, post.translated_base_path(lang), os.path.join(self.site.config['OUTPUT_FOLDER'], post.destination_path(lang)) 19:18:07 <erdgeist> do you know who wrote the post-list directive feature, so I could ask directly? 19:18:42 <ChrisWarrick> @udono in github, Udo Spallek 19:18:45 <ChrisWarrick> on* 19:18:49 <erdgeist> thanks 19:19:29 <ChrisWarrick> good luck with all that! 19:20:19 <erdgeist> thanks a lot. I duly took notes of all your suggestions :) 19:20:51 <ChrisWarrick> dependency recording could do it, actually 19:25:21 <ChrisWarrick> wah 19:25:30 <ChrisWarrick> the actual problem appears when you start recording dependencies 19:31:06 <ChrisWarrick> All this happens because the dependency files are not used in the original generation and you get a nice little race condition 19:31:17 <erdgeist> yep 19:31:50 <ChrisWarrick> actually, there is one hack you could try 19:32:10 <erdgeist> I am just looking into how from post_pages the tasks are generated 19:32:24 <ChrisWarrick> by time of post creation 19:32:56 <erdgeist> Hmm, I could re-sort them 19:32:59 <ChrisWarrick> nah 19:33:10 <ChrisWarrick> just set the date of your index post to 1970-01-01T00:00:00+00:00 19:33:12 <ChrisWarrick> that will be enough 19:34:52 <ChrisWarrick> the timeline is ALWAYS sorted in reverse chronological order, that’s guaranteed 19:35:21 <erdgeist> getting the same errors 19:36:10 <ChrisWarrick> you must have modified some code somewhere 19:36:47 <ChrisWarrick> try nuking the cache and database just in case 19:37:00 <erdgeist> database is .doit.db? 19:38:00 <ChrisWarrick> yes 19:38:41 <ChrisWarrick> I just successfully produced a page with post.text() in the template and no problems 19:39:18 <ChrisWarrick> I modified post-list to register dependencies on the source file (.rst) and it works fine 19:39:53 <ChrisWarrick> And I also get dependencies for the cache/ files from post.text() apparently 19:41:30 <erdgeist> ChrisWarrick: can I just send you the source for my page and you try it out? It's just one MV 19:41:33 <erdgeist> MB 19:41:35 <ChrisWarrick> go on 19:42:53 <erdgeist> https://erdgeist.org/erdgeist.org.tbz 19:45:38 <ChrisWarrick> erdgeist: 20:33:10 <+ChrisWarrick> just set the date of your index post to 1970-01-01T00:00:00+00:00 19:46:21 <erdgeist> -rw-r--r-- 1 erdgeist 503 221 1 Jan 1970 stories/poetry/index.rst 19:46:50 <erdgeist> maybe I forgot to use -p on my tar command 19:47:03 <ChrisWarrick> oh 19:47:08 <ChrisWarrick> you set the file system date 19:47:19 <ChrisWarrick> I meant the .. date: meta field of your post 19:47:31 <erdgeist> oh, how embarrassing 19:48:03 <ChrisWarrick> Nikola itself doesn’t care about file dates, they cannot be trusted in most cases 19:48:54 <erdgeist> much more betterer 19:49:05 <erdgeist> thanks! 19:49:21 <ChrisWarrick> again, a lot of things done in this is using hacks 19:49:58 <ChrisWarrick> one day we could sort site.timeline by /dev/urandom and break everything (that’s nonsense, we won’t) 19:50:07 <ChrisWarrick> bonus question: what magic sorcery did you use to set the file creation date? 19:50:47 <erdgeist> touch -t 197001010000 index.rst 19:50:54 <erdgeist> posix voodoo 19:52:22 <ChrisWarrick> I always forget touch can be used to do this 19:52:48 <ChrisWarrick> in fact, it’s its main function, and creating new files only is not the intended functionality 19:53:24 <erdgeist> hehe 19:53:51 <erdgeist> I usually type "touch .pnwed" on other ppls servers 19:58:52 -GitHub[nikola]:#nikola- [nikola] Kwpolska pushed 2 new commits to master: http://git.io/pf3W 19:58:52 -GitHub[nikola]:#nikola- nikola/master df2eb6a Chris Warrick: record some deps in post_list... 19:58:52 -GitHub[nikola]:#nikola- nikola/master a4ff446 Chris Warrick: fix #1629 -- document different input formats... 20:04:12 -GitHub[nikola]:#nikola- [nikola] Kwpolska pushed 1 new commit to master: http://git.io/pfZU 20:04:12 -GitHub[nikola]:#nikola- nikola/master 41356a1 Chris Warrick: fix #1625 -- ignore post meta in two-file posts... 20:13:32 -travis-ci:#nikola- getnikola/nikola#4663 (master - a4ff446 : Chris Warrick): The build was broken. 20:13:33 -travis-ci:#nikola- Change view: https://github.com/getnikola/nikola/compare/37ebddc5af0c...a4ff4460af79 20:13:33 -travis-ci:#nikola- Build details: http://travis-ci.org/getnikola/nikola/builds/53389001 20:15:44 -travis-ci:#nikola- getnikola/nikola#4664 (master - 41356a1 : Chris Warrick): The build was broken. 20:15:45 -travis-ci:#nikola- Change view: https://github.com/getnikola/nikola/compare/a4ff4460af79...41356a1ee9ad 20:15:45 -travis-ci:#nikola- Build details: http://travis-ci.org/getnikola/nikola/builds/53389441 21:02:31 -GitHub[nikola]:#nikola- [nikola] Kwpolska pushed 1 new commit to master: http://git.io/pfS8 21:02:31 -GitHub[nikola]:#nikola- nikola/master 9762df2 Chris Warrick: flake8... 21:11:03 -travis-ci:#nikola- getnikola/nikola#4665 (master - 9762df2 : Chris Warrick): The build was fixed. 21:11:04 -travis-ci:#nikola- Change view: https://github.com/getnikola/nikola/compare/41356a1ee9ad...9762df25096a 21:11:04 -travis-ci:#nikola- Build details: http://travis-ci.org/getnikola/nikola/builds/53395474