IfcOpenShell - project and repo structure - modernise/split up/clean up
I would like to start a discussion about the project/repo/folder structure of the IfcOpenShell project and possible ways to modernise and maintain a concise project structure.
Currently the repo is a big bag of everything. Literally. The main parts are IfcOpenShell and BlenderBim, both massive projects on its own. And there are more smaller projects which depend on IfcOS, but are not necessarilly a vital part of the core project. More like smaller utilities or additions, even just explorations into other areas. Some source code is even unmaintained or stale.
I would like to propose a split of this monorepo into smaller units. I think this would greatly help in the general work on both projects and make these parts better maintainable.
- finding your way around the project is challenging. There is so much going on right now that is is difficult to grasp the interconnectivity between the single parts
- most issues and commits happen in regards to BlenderBim. It is hard to keep up with changes to IfcOS-Core or other parts of the project
- The branching structure is very unusual. The main work happens in branch
v0.7.0(SemVer!) not in a main or master branch, although these exist too - The folder structure is odd. Everything lives in
/src, even if it’s just examples or tools. Dead folders are kept around, even though they are preserved in the git history.
What do you think? Do you (the project maintainers) want to touch this topic and are you open for PR’s?
Comments
Just about to go to sleep, but this has been brought up a couple years ago here: https://github.com/IfcOpenShell/IfcOpenShell/issues/1497
Thanks for the link. I think I have seen this issue before. Yet I think it is time to revive the discussion.
Your (and @aothms) points are mostly about:
These arguments make it seem like IfcOS as a library only exists for the benefit of BlenderBim. It leaves users with other use cases and projects out.
Ones which do not need BlenderBim as a GUI but use IfcOS as a dependency on their own projects or in other context, lets say within a Jupyter notebook or a small stand alone utility program. Or even an alternative desktop authoring app based on IFC. Sure, I can use IfcOS as a dependency, but I also get a lot of excess luggage too.
I’m not so really concerned about the notification noise within the repo, as mentioned in the issue. I would rather argue in the context of IfcOS as an independent library, just as it says on the tin (ifcopenshell.org). With feature parity of the C++ and the Python API.
One thing of the tight dev workflow between both projects is an incredible advancement in the Python API. But I have the feeling the the C++ API lacks behind.
So in short, I would favour having a IfcOS library which does one thing (or a well defined set) and does it well. I think this would also benefit the spreading of IfcOS within the wider aec eco system.
Side note:
Recently there was an interesting article about the core philosophy of the Unix program design. Libraries and apps, which do on thing and do it well.
The IfcOS repo currently IMV is not well defined. It looks more like a huge playground, where everything is thrown in. This is not beneficial for longtime maintenance.
Here are some notes on the IfcOS repo I made while exploring the code base. These might give an outside view of a newcomer to the source base and trying to understanding it as opposed to someone who is intimate familiar with it.
notes on the root folder structure:
/toolsfolder to make the use case clearerREADMEfile to help understanding the content and usagesrc/ifcblenderexportfolder.ifcand.zipfiles and perform tests on them, but partially also relies on external Blender installednotes on the
/srcfolder:/root, most files lack an explanation or comments on what they doREADME/ifcopenshell-python - Python bindings and high-level API for IfcOS
/toolsfolder under/root, as this is used to create IfcOS C++ code, but not for the actual use of IfcOS/ifcparse - not yet explored, missing
READMEREADME/root/viewerdirectoryReadme.mdwith use-case/svgfill - a git submodule to https://github.com/IfcOpenShell/svgfill -> this should be located under a
/root/thirdpartyor/root/externalfolder to make the dependency relation clearparts of the c++ code are not build as libraries, they are included via
#include "../ifcgeom/IfcGeom.h", but should be included as#include "ifcgeom/IfcGeom.h"to make them path independentThere seems to be a few comments here. Let's start with the things I 100% agree upon:
I'd like to clarify a few things:
I'd also like to emphasize a few points of view which I've come to really appreciate over the past few years:
I don't see how splitting repos helps users who only use IfcConvert, or only use IfcOpenShell-Python. We already bundle and package them separately, so isn't that sufficient?
Might I propose some low risk smaller scale changes that should have an immediate benefit and address some of your concerns first?
That's probably the easy stuff. I'd then perhaps propose to start reorganizing non source code, stuff like build scripts, examples, docs.
Finally, the hard stuff: actually moving source, improving branching, adopting semver, etc.
If this sounds like a practical plan, would you be interested to help contribute? Especially in the C++ side as you can see we are really short on resources in improving docs and examples. There's also a lot of work done in the v0.8.0 branch and refactoring in progress.
Thank you for that extensive explanation, very insightful.
I see from the project history that a lot of the structure makes sense, especially in light of its organically growth. Even if I don't agree with everything from a current view onto the project, I will leave it at that. Yet I would appreciate if the project stays open to continuously challenge its status in terms of structure and organisation. I certainly will nag you periodically.
This quote highlights my confusion with the unusual branching model very well. The first look into this branch says "This branch is 332 commits ahead, 1380 commits behind v0.7.0.". Branch v0.7.0 seems current, branch v0.8.0 might be in dev but the git status and commit history tells otherwise. So how would I know?
I like your proposed steps for a refactoring in small steps. Let's start with that, I'm happy to support in ways I'm able to. Some remarks though:
I think you talk about the extensive documentation of modules. This is very important, especially in terms of deeper explanations of internals and architectural decisions, examples, use-cases etc.
But if I explore the code base in an editor and have a subfolder open (e.g. /ifcconvert or /ifcdiff), I would have to open an external browser to the docs or change folder to ../ifcopenshell-python/docs/ifcconvert.rst. Not very intuitive.
I talk about a short and concise
Readme.mdto give a quick overview of the contents and purpose of a folder/module so I can find my way around. Good examples are the the Readme files in /ifc2ca or /ifcgeomserver.Yes indeed, I do. In fact I already tried, but the response so far was rather poor -> https://github.com/IfcOpenShell/IfcOpenShell/pull/2393
A little update on this that the 4 easy proposals are now implemented. The next thing on the list is to adopt some form of semver, better versioning and improved distribution.