Unification of BlenderBIM and IfcOpenShell web experience

Hi Community,

As you're probably aware, BlenderBIM is being developed as part of the IfcOpenShell repo on github. The experience on the web for both projects is not very coherent. Since mostly http://ifcopenshell.org is horribly outdated I think it makes most sense to get ifcopenshell.org to adopt the BlenderBIM theme and go from there.

None of this is my expertise, are there people interested in helping out with this or advise a bit? I had the following ideas in mind:

v1

  • Get BlenderBIM theme and convert to flask/jinja2 (cc @Moult)
  • Update ifcopenshell.org website (flask) to use new template
  • Remove outdated content: IfcBlender IfcMax, ...
  • First level menu item to blenderbim.org (and from there back to ifcopenshell.org)

v2

  • New content: IfcPatch, IfcDiff, ... (see forums for logos)
  • New content: some sort of carousel with example usage
  • Revisit ifcopenshell logo and pairing with text

v3

Thanks,
Thomas

MoultJesusbillSigmaDimensionsCadGiru
«1

Comments

  • Ping @tetov maybe your cup of tea? :)

  • edited April 2021

    That's great @aothms! Regarding documentation I recently came across the Executable Book Project that was used in this project that caught my attention (COFEA Initiative and which by the way I have to document somewhere in our wiki).
    I consider it a great theme for documentation, seems to work great for mobile as well and it is always based on Sphinx.

    krande
  • We have discussed sub forums before and I don't see any special problem with it. I haven't supported it earlier because it divides the discussion into fragments. It's nice and useful to see what everyone else is doing.

    With that said there is talk of switching to a different forum system. So maybe now is not the time to make a subforum - but just mix it into the main forum. We have some tags and I sometimes merge threads if they deserve it - so it's not a completely disorganized mess even though there are a lot of different things being discussed.

  • Took a quick look. Are are both built using Sphinx?

  • @tetov both https://blenderbim.org/docs/ and https://ifcopenshell.github.io/docs/ use Sphinx - yes, they have the same source. The former is something I set up a long time ago without any automated updates, and the latter I think has automated updates. Just theme differences.

  • I'll start on it this week :).

    MoultJesusbillbitacovirCadGiru
  • Bump :)

  • Haven't forgotten, just been busy. I'll get started this week or next.

    Moultduncan
  • edited June 2021

    @Moult @aothms:
    First off; where's can I find the source code for the icfopenshell website (ifcopenshell.org)? :)

    We currently have two "landing sites" and two auto-generated docs sites.
    blenderbim.org is generated with mustach templates and some makefiles, and skeleton.
    I suggest that I convert the mustach templates to a jinja/sphinx template and use that for index and handwritten pages on a Sphinx site. We can keep using stock template(s) for the autogenerated docs.

    In short; integrate generation of landing sites into Sphinx using new jinja templates/css based on blenderbim.org.
    What do you think?

  • Btw, just bumped http://git.sevenstrokes.net/?p=blenderbimorg.git;a=tree with some changes I forgot to sync. If you give me some SSH keys I can give you access, otherwise just go from a snapshot and let's start afresh with some shared code on a more public repo (e.g. Github).

    More than happy to switch to Jinja/Sphinx. I haven't used Sphinx much, but I assume it allows both auto doc generated stuff, as well as totally custom HTML (e.g. some of the pages are quite custom, which we should keep).

  • Thanks! You can copy my key from /home/osarchorg/.ssh/authorized_keys if that works for you, otherwise I’ll use a snapshot :).

    I was thinking of doing custom html through templating where RST + jinja doesn’t cut it :).

  • Bump

    @tetov said:
    @Moult @aothms:
    First off; where's can I find the source code for the icfopenshell website (ifcopenshell.org)? :)

  • Ah ping @aothms for that one. Also sorry for the delay on my end but I've supposedly given both your keys RW access to the blenderbimorg git repo.

  • Yeah sorry for the delay, it needed some cleaning before putting it online. The code is here now https://github.com/IfcOpenShell/website Let me know if you have questions or comments

    Thanks goes to Johan Luttun for implementing the functionality to extract the builds from the IfcOpenBot github repo (which unfortunately had to be disabled as the builds were falling so far behind that we couldn't catch them in the API rate limits)

  • No worries, thanks for sharing the source. @aothms. That looks like it can be adapted into a Sphinx setup quite neatly :).

    I started to dig into this two days ago, but I got a bit sidetracked getting sphinx to run for ifcopenshell..

    My first thought was to run sphinx on commit to branch v0.6.0 for ifcopenshell and blenderbim, with landing page etc under the toplevel and API docs in /docs.

    The run duration of the sphinx/breathe/exhale/doxygen makes build on every commit a bit wasteful. Maybe we run it once a day (if there are new commits since yesterday).

    We could of course decouple the generation of pages and API docs, but that adds unnecessary complexity in its own way.

    In my view my next task is as stated in one of my earlier posts in this thread: Create a jinja template that is as close as possible to the looks of blenderbim.org and tweak the landing page sources to use that theme.

    However, I’m going away for two weeks starting tomorrow, back again 22 July. I will not bring my laptop, so this will have to wait a bit more. If I find time I’ll do some planning and research into possible solutions for “continuous documentation”.

  • The run duration of the sphinx/breathe/exhale/doxygen makes build on every commit a bit wasteful

    Yes it's crazy. For the C++ bit it takes hours to nearly a day.

    as close as possible to the looks of blenderbim.org

    I'm sure you can take a lot of liberty in making improvements where you see fit.

    We could of course decouple the generation of pages and API docs, but that adds unnecessary complexity in its own way.

    Maybe we should give this some thought. In the end we'll likely have (a) end user manual (b) code documentation (c) tutorials (d) website. I didn't give this a lot of thought, but it sounds like these all shouldn't be one monolithic thing.

    back again 22 July

    Enjoy your holiday!

  • Awesome! And yes please feel free to make any changes you see fit. Fresh eyes are very valuable. I think the code docs / end user manual should be separate from the site, but that's just me :)

  • @aothms said:
    Maybe we should give this some thought. In the end we'll likely have (a) end user manual (b) code documentation (c) tutorials (d) website. I didn't give this a lot of thought, but it sounds like these all shouldn't be one monolithic thing.

    @Moult said:
    […] I think the code docs / end user manual should be separate from the site, but that's just me :)

    Thanks for your input!

    I’m very used to python projects with a Sphinx setup for all web presence. See compas_fab, (doc source, build action).

    Putting A, B, C together (end user manual, code documentation & tutoriaks) is nice because you can link to generated docs easily (or provide tooltips etc), you can run tests for codeblocks in tutorial/manual and write your tutorials/manual in RST or MD. Having all documentation sitting close to source code makes it easier to update both at the same time.

    That’s why I think making ABC a monolith makes a lot of sense. But decoupling ifcopenshell docs and blenderbim docs would probably be for the best since we can build blenderbim docs on every commit.

    How about this:
    1. The current monorepo (ifcopenshell/ifcopenshell) as is, with scripts/github actions/docker image for building ifcopenshell and blenderbim A, B & C
    2. Two repos where main/master holds sphinx generated html/js from source files in the monorepo (ifcopenshell/ifcopenshell).
    3. Two website repos where branch master/main holds source files and branch gh-pages holds static html/js generated with github actions and hosted by github (D). Respective repos described in last step are brought in as submodules and end up on the gh-pages branch under “docs/“ tying ABC to D.

    The repo/sites in 3 should be powered by a static html generator to allow templates and easy editing by everyone. Sphinx would make sense to keep it consistent, or another more versatile generator with support for jinja templates. The template for this part could eventually be used for ABC as well.

    We could of course also use a trendier generator with support for React (think Gatsby) but I’m guessing that Python will be more familiar to devs active in ifcopenshell/ifcopenshell.
    Thoughts?

  • Sounds good to me, let me know which repos to create.

    Some thoughts on static site: Currently there is some python logic in ifcopenshell.org to fetch the latest builds from github commits. It's actually pretty awful. The build system (IfcOpenBot) is another slow and complex process. So we need some service where information on IfcOpenShell builds is collected and fetched. Alternative is that IfcOpenBot pushes to the repo to trigger an update of the static site (like update a json file somewhere that is rendered by the static site). Thoughts?

  • @aothms: I think IfcOpenBot builds and somehow publishes available downloads, through something that can then be parsed by site generator. We can trigger a build using a github webhook. It's quite easy to use a webhook to start another repo's github action :).

    Moult
  • Webhook, so not a commit? Anyway, I trust you on this. let me know what I can do in terms of creation of repos and adaptations to build scripts :)

  • Hm, I thought the release pipeline looked like this:
    1. Commit is selected for release and tagged with version no.
    2. Someone (or script triggered by the tag) runs buildsscripts in IfcOpenShell/IfcOpenBot and uploads artifacts to S3 and commits something(?),
    3. Flask site at IfcOpenShell/website scans commits looking for IfcOpenBot as commiter and product in commit body and builds pages based on that. (How often does flask regenerate this?)

    What I would propose is a github action bridging 2 and 3. It just looks at commit (or tag?) and checks if it warrants an update of release artifacts on website. If the website needs to be updated it sends a HTTP POST payload to website repo triggering Flask (or static site generator) to regenerate its info about releases. This is something I've seen used a lot to have a CMS or a comments system trigger a rebuild of a static website without commits to the website's repo.

    I haven't fully understood the IfcOpenBot's role, especially with the commits, but one thought would be to have it run a build under certain conditions (or triggered manually) and as the last step after uploading to S3 send a POST request to website repo to regenerate. And the website could base it's information of available artifacts on a the contents in the S3 bucket, by querying the S3 api for bucket contents and parsing file names. Would that make sense?

  • I thought the release pipeline looked like this

    Mostly correct.

    IfcOpenBot doesn't look at tags, it just randomly pulls and then starts building HEAD. We have very few tags in the ifcopenshell repo.

    IfcOpenBot indeed uploads to S3, but doesn't do a commit somewhere. It uses the github API to do a commit comment.

    one thought would be to have it run a build under certain conditions (or triggered manually) and as the last step after uploading to S3 send a POST request to website repo to regenerate. And the website could base it's information of available artifacts on a the contents in the S3 bucket, by querying the S3 api for bucket contents and parsing file names. Would that make sense?

    So instead of doing some custom POST message somewhere and have the site generator query the S3 API. Why not have IfcOpenBot do a commit to the website repo? That would also trigger a rebuild and doesn't require that much logic in the site generator script. Downsides would be the noise from the amount of commits maybe to the website repo? But it's more vanilla git, less proprietary APIs, therefore more portable?

  • Static website should not require any change at build time.
    Maybe a "latest release" static symlink on buildbot/repo side so url will never change, and a trick to retrieve version string through js are enough and will simplify things on website side.

  • @aothms said:
    Why not have IfcOpenBot do a commit to the website repo? That would also trigger a rebuild and doesn't require that much logic in the site >generator script. Downsides would be the noise from the amount of commits maybe to the website repo? But it's more vanilla git, less >proprietary APIs, therefore more portable?

    Agreed. IfcOpenBot can commit a machine readable file with url and some metadata which can then be picked up by static website generator. Way better!

  • We're in the process of switching branches in IfcOpenShell. Upgrading from 0.6 to 0.7. So getting the website updated and the fresh builds listed on the website is getting a bit higher priority.

    In the meantime I've also experimented myself with static site generation for the IfcOpenShell academy https://academy.ifcopenshell.org/ https://github.com/IfcOpenShell/academy/tree/main/posts I just opted for Nikola because it's python based and because it supports notebooks for posts. Nikola doesn't seem very suitable though for the main web experience.

    What can I do to speed up the progress on this?

  • Bump - @tetov, can we help in any way?

  • Hi! I'm not the most experienced web developer but I've done a couple of things. Let me know if I can help in any way.

    theoryshaw
  • Hey @Pagody ! That would be awesome! Does the original post make sense to you? Do you have any experience in Sphinx (as some of the project documentation is currently generated with Sphinx, and we want to expand on it)? It's a pretty simple static site for the most part apart from Sphinx I believe.

  • @Moult said:
    Hey @Pagody ! That would be awesome! Does the original post make sense to you? Do you have any experience in Sphinx (as some of the project documentation is currently generated with Sphinx, and we want to expand on it)? It's a pretty simple static site for the most part apart from Sphinx I believe.

    The original post makes sense to me. I have a familiarity with the basics of Sphinx, but if that's not enough I'm more than happy to study and learn. I'm transitioning to a more programming oriented career so anything that can help me build experience and knowledge would be very welcome. My background is mostly Python and the web developer basics (html, css, some javascript).

Sign In or Register to comment.