IfcOpenHouse - Step-by-step tutorial with the IfcOpenShell Python API

cvillagrasacvillagrasa
in General

Hi!
I've been recently needing to level up my skills with the IFC schema, from the coding standpoint, because I feel that lots of useful things can be accomplished from that angle. Along those lines, I found a couple of random posts about an "IfcOpenHouse", and I decided to have a look into it in order to learn more. It turns out that it was an @aothms project from 2012, in which he demonstrated the capabilities of IfcOpenShell to programmatically generate a house. And all of that in C++! Huge feat for a time in which most of the industry didn't even understand the word "BIM" ?.

Link to the tutorial

So some weeks ago, I thought that it would be cool to try to recreate the same model with the high level API as a learning exercise. You can find the result as a step-by-step tutorial under the following link:
https://cvillagrasa.github.io/IfcOpenHouse/generation.html

Appart from IfcOpenShell, I've used nbdev to autogenerate a Quarto website + plain Python scripts, IFC.js for viz, and some custom code to make all of those pieces stick together.

The project consists of a Jupyter Notebook environment with some incorporated assistance to experiment with IFC models. The previous link points to the static website, for ease of access, from which there are further links to the actual notebook and the rest of the repository, if interested.

Questions that have arised

It is worth noting that at the moment, the project is totally experimental and should by no means be considered a correct usage of IFC nor the IfcOpenShell library. In fact, I've even placed some questions with my own doubts into Python comments. @Moult if you have time at some point, search in the link for "#q" and you'll come across those questions in context, although some of them are just mere suggestions. They are also listed as follows:

  • q1: Data-scientish notation as suggested below, context1, context2.
  • q2: Is project.assign_declaration(file, definition=project, relating_context=project) necessary? what does it do? I copied it from some snippet I found.
  • q3: For clarity, should there be a context.add_subcontext call?
  • q4: For placements, should there be an option to just give X (RefDirection) and Z (Axis), and don't deal with any matrix?
  • q5: Should there be a geometry.add_extruded_representation on a higher level than shape_builder? Accepting optional width/depth, or radius, or a profile.
  • q6: Shouldn't there be an elegant solution by the API to avoid monkey-patching?
  • q6b: Calls to an add method, followed by another call to an edit method, like when adding psets, could be simplified into one call?
  • q7: How are IfcRelConnectsPathElements with ConnectionGeometry made?
  • q8: Any viewer in which to check advanced BREPs with IfcBSplineSurfaceWithKnots?
  • q9: I removed usages, as they were in the original IfcOpenHouse. The validator was complaining, and it seems that in "modern IFC", usages go together with types. Should I add types? types + usages? leave it as it is?
  • q10: When building doors or windows with geometry.add_door_representation and geometry.add_window_representation, will it be possible to apply different styles to different "subrepresentations"? As in a wooden lining + glass panel, or a metallic door knob.

Below, there are some specific features which can be found in the notebook and deem some explanation:

Data Science notation

I'd love if the API leveraged the benefits of static signatures in the future. In a notebook environment, that would mean that running an API call followed by one question mark would show the help, and followed by two question marks would directly show the code. For now, in order to avoid calling methods as strings and have a conveniently short alias for the library more identifiable than "run", I created a temporary helper class which is imported as "ios" (short for IfcOpenShell).

Automatic garbage collection of overwritten entities

Envision for instance the following scenario: a cell of the notebook creates a wall entity as follows:

my_entity = ios.root.create_entity(file, ifc_class='IfcWall')

(Yes, I'm using a non-standard ios in a data-scientish way, to avoid strings and shorten commands to a significant alias, but that's another story).
Let's say that we run that cell, so that the variable my_entity now holds the newly created instance for the wall (say step ID #20)... but for some reason we regret, and rerun the cell creating a column instead:

my_entity = ios.root.create_entity(file, ifc_class='IfcColumn')

What now happens is that the variable my_entity points towards the newly created column (say with step ID #21), but the previous wall #20 doesn't get deleted from the file. This is logical from a scripting standpoint, but feels totally counterintuitive when interacting within a notebook environment. In order to be able to experiment without constantly polluting the IfcOpenShell file with unwanted instances, an audit hook provides some level of garbage collection. In the previous example, the wall entity with step ID #20 will be removed from the file when overwriting the Python variable pointing to it. It has also been successfully tested by changing the angle of the roof and rerunning all depending cells, and other similar tasks. Nevertheless, bear in mind that this is a highly experimental code and the door is open to unexpected results, so it shouldn't be used in production under any circumstances.

IFC.js viz

Since Jupyter Notebooks run in the browser, it is possible to use IFC.js to visualize our IFC model in real time, which altogether with the mentioned garbage collection, really allows for a truly interactive session. In the notebook, the file is parsed as a string with IfcOpenShell, and then retrieved from the Python session with JavaScript, allowing for a smooth all-in-memory workflow, not requiring to save and read from disk as an intermediate step.
Some issues I was having is the current version of web-ifc-three not reading IFC4x3 (although it seems that this will change soon), and also web-ifc-three not showing transparency, at least for styles directly assigned to representations. This was worked around by some custom code for traversing the IFC entities in order to check for transparent styles.

Summary

So now, there are some open questions, some obscure sorcery with non-standard library aliases, Python audit hooks... so I guess there is still some room for changes. @aothms I'll be happy to link it to the academy repo when you judge it's ready (beware that as I pointed out, at the moment the code has 10 questions inside the comments).
That was all for now! thank you for reading this authentic IfcSlab ?

CoenAcebrunopostleGorgioustheoryshawNigelMoultArvJesusbillMaartenFroughand 2 others.

Comments

  • aothmsaothms

    This is obviously amazing, thank you for this!

    q8: Any viewer in which to check advanced BREPs with IfcBSplineSurfaceWithKnots?

    Did you try BlenderBIM? ;)

    q11 What is Size3D / 2d. I would propose to keep the tutorial as vanilla-ifopsh as possible. If there are unergonomic parts it's a sign that we might need to extend the API. The API is mostly designed with a modelling application in mind, not with coding, so this is truly a great exercise.

    q12 This immutability came also up in the visual programming discussions we had past year. The file-container-model doesn't work well in more pure functional settings, such as this or a visual programming. Should we invest effort in a mechanism that allows invoking API methods on pure-python immutable dicts-of-dicts kind of data structures?

    q13 I must say though that I had hope the code would be shorter. It's still a massive read to plough through. Maybe we should also spend some time here and think about approaches of reducing the amount of code involved to generate a simple building model like this. The first thing I can think of is to be much more "geometric" as opposed to explicit. Start from a rectangle. Define walls for the edges in one run. Offset the rectangle to create the footing. Etc. But that would basically mean building another level of higher api on top of we already have. It would also have a hint of visual programming maybe. This is obviously not for now, but somehow this exercise tells me we're not quite there yet in terms of providing an API that eliminates underlying complexity of IFC to the extent that it becomes amenable to humans. No critique at all.

    MoultcvillagrasaGorgious
  • cvillagrasacvillagrasa

    @aothms said:
    This is obviously amazing, thank you for this!

    A million thanks!

    Did you try BlenderBIM? ;)

    Indeed, it shows nothing, but I didn't know it was supposed to work with advanced BREPS already. See BlenderBIM left / IFC.js right:

    So IFC.js does attempt to draw some random border. That can perfectly mean that the surface is not correctly generated...

    In BlenderBIM, even if not displayed, the representation is correctly recognized, though:

    q11 What is Size3D / 2d. I would propose to keep the tutorial as vanilla-ifopsh as possible. If there are unergonomic parts it's a sign that we might need to extend the API. The API is mostly designed with a modelling application in mind, not with coding, so this is truly a great exercise.

    I thought that there was already a significant amount of concepts for a beginner, and tried not to overwhelm too much. The Size2D thing maybe doesn't dramatically improve readability (obj.z Vs obj[2]), but at some point I removed type hints and thought it would help a bit. Also, the __repr__ feels more intuitive if someone starts exploring the variables within the notebook:

    On the other hand, maybe something like the colour RGB thing would indeed fit into style.add_surface_style. And the same with placement matrices and clippling planes.

    q12 This immutability came also up in the visual programming discussions we had past year. The file-container-model doesn't work well in more pure functional settings, such as this or a visual programming. Should we invest effort in a mechanism that allows invoking API methods on pure-python immutable dicts-of-dicts kind of data structures?

    If you're thinking of a way in which something like this could be done, you're already a thousand km ahead of me. Within a notebook environment, upon a given cell execution, how can you know from the API side if something is being overwritten? The only hint is the Python global variable. You could do something such as creating a class with a custom __setattr__ (or equivalent magic in C++) to keep track of declarations, but then you'd need to assign your variables as my_custom_class.my_variable = value, which feels ultra unintuitive and is something I wanted to avoid.

    So, my solution was to design this audit hook, which checks 1) whether the variable being assigned is being overwritten, and 2) whether it is an IfcOpenShell entity instance, in which case it momentarily disables the hook to avoid infinite loops, and proceeds to assign the variable to None. Lastly, it removes the entities from the file with a variant of @Moult 's ifcopenshell.util.element.remove_deep2 divided into two steps and without batching. I deemed batching wasn't very important for the typical file sizes that will be managed in notebooks, but I guess it could also be added back.

    As for visual programming, I also see the same fundamental problem than with a Jupyter Notebook. How are you thinking this could be solved from the API side?

    q13 I must say though that I had hope the code would be shorter. It's still a massive read to plough through. Maybe we should also spend some time here and think about approaches of reducing the amount of code involved to generate a simple building model like this. The first thing I can think of is to be much more "geometric" as opposed to explicit. Start from a rectangle. Define walls for the edges in one run. Offset the rectangle to create the footing. Etc. But that would basically mean building another level of higher api on top of we already have. It would also have a hint of visual programming maybe. This is obviously not for now, but somehow this exercise tells me we're not quite there yet in terms of providing an API that eliminates underlying complexity of IFC to the extent that it becomes amenable to humans. No critique at all.

    Yes, definitely, if everything is accounted, there are more lines of code in this Python version than there were in the original C++ version ?.

    I'm not entirely sure that turning more geometric is necessarily going to save a lot of LoC, but I'll give it a thought. My feeling is that the Python API is very useful, but still a bit low level. That's not a critique at all to the API, but maybe it means that with time, other higher level APIs will get built on top of it. However, with higher levels of abstraction come greater distances to the Schema, different assumptions made by different people, etc. implying that those future higher level APIs will have a tough task in pleasing everyone.

  • aothmsaothms

    It processes fine in IfcConvert (you can check it's the new version as it has the fancy door handle), so maybe this is sth for @Moult to look at..

    The Size2D thing maybe doesn't dramatically improve readability (obj.z Vs obj[2]), but at some point I removed type hints and thought it would help a bit. Also, the repr feels more intuitive if someone starts exploring the variables within the notebook:

    Blender's mathutils.Vector class is nice in that sense, because you can do both. Also C++ libs like Eigen allow both [0] (0) and .x. I don't know why numpy never went as far. In this case I'm not sure if it helps, I didn't know that it was just a dataclass for example, I though maybe it does a lot more magic like overloaded operators.

    As for visual programming, I also see the same fundamental problem than with a Jupyter Notebook. How are you thinking this could be solved from the API side?

    Essentially, it's just immutability, never modify anything, just return updated copies. But this is actually where we are in a pretty good position with the API at least, because there is a universal entry point for all API calls. So what we can do there is before invoking the actual method, create a copy (file.from_string(file.to_string())), invoke the API method on that, and return the newly created and updated file. Performance would probably be abysmal, but it could be an idea to at least experiment with it? You'd likely have to be really careful though that instances are actually from the correct file. So maybe you'd just use guids..? I don't know.

    My feeling is that the Python API is very useful, but still a bit low level. That's not a critique at all to the API, but maybe it means that with time, other higher level APIs will get built on top of it.

    I agree. I would characterize it as that this level of API is perfect at maintaining a valid and idiomatic IFC graph (which is a huge feat).

    I'm not entirely sure that turning more geometric is necessarily going to save a lot of LoC, but I'll give it a thought ... However, with higher levels of abstraction come greater distances to the Schema

    True. Especially if you the structure of the it around a foreign concept like this geometry-first approach I was describing. Maybe that is really only valid in visual programming. In this case there should be no tight coupling, but it could be possible to structure the code like that.

    Maybe for inspiration: https://dev.opencascade.org/doc/overview/html/occt__tutorial.html

  • cvillagrasacvillagrasa

    @aothms said:
    It processes fine in IfcConvert (you can check it's the new version as it has the fancy door handle), so maybe this is sth for @Moult to look at..

    It doesn't work on my end. Maybe you've used the tesselated version with PythonOCC? (It's selected by default because it's the one that works)

    I'll need to revisit the advanced BREP generation. It's difficult, though, since there aren't a ton of examples out there to compare with.

    Blender's mathutils.Vector class is nice in that sense, because you can do both. Also C++ libs like Eigen allow both [0] (0) and .x. I don't know why numpy never went as far. In this case I'm not sure if it helps, I didn't know that it was just a dataclass for example, I though maybe it does a lot more magic like overloaded operators.

    Ah yes, standalone mathutils could be a good alternative. I've had some headaches with it and nbdev on GitHub Actions, one needs to be careful on the Python version. But anyway, the door and window generation wouldn't work without that dependency in any case, so I might change the Size2D thing to mathutils ?.

    Essentially, it's just immutability, never modify anything, just return updated copies. But this is actually where we are in a pretty good position with the API at least, because there is a universal entry point for all API calls. So what we can do there is before invoking the actual method, create a copy (file.from_string(file.to_string())), invoke the API method on that, and return the newly created and updated file.

    I don't understand it. How does it work without needing a new file / immutable dict variable at every given step? Say, in the scenario where we have a cell that adds a wall, and we run that cell again and again.

    As for performance, I assumed the audit hook option would also be abysmal, but I just did a test and speeds are almost the same:

  • cvillagrasacvillagrasa

    Little update because those times were suspicious: the Python audit hook doesn't work but one time on %%time, %%timeit or inside for loops...

  • cvillagrasacvillagrasa

    @cvillagrasa said:
    I'll need to revisit the advanced BREP generation. It's difficult, though, since there aren't a ton of examples out there to compare with.

    I sorted it out! Indeed, there were some problems with the edge loop. The IfcBSplineSurfaceWithKnots works out of the box both in BlenderBIM and IFC.js ?:

    I've left the tesselation method as default, though! since I feel it's more robust and will work in more viewers.

    CoenArvJesusbill
  • cvillagrasacvillagrasa

    Hi again, I'm currently having some doubts regarding question 7, is there any place where I can learn more about connecting elements / connecting paths for walls? why don't connections between sloped slabs and walls have ATEND or ATSTART? in other words, why are they "mere connections" and not paths?

    Also, do ConnectionGeometry work in BlenderBIM? or is something that the Schema provides but no one uses? Thanks

Sign In or Register to comment.