Update on Project Management features - Construction Scheduling and Cost Scheduling.
After 2 weeks of work with @Moult , we are really excited to annouce that we have now implemented in blenderBIM and the ifcopenshell.api, the following features:
Authoring of IfcWorkPlans, IfcWorkSchedules, IfcTasks, IfcTaskTimes entities,
Authoring of IfcCostSchedules, IfcCostItems entities.
Assignement of Task Successors and Task Predecessors relationships.
Generation of Gantt Charts from an IfcWorkSchedule.
Importing an IfcWorkSchedule/from a P6 XML export
Assignement of IfcProducts ( e.g building elements) to IfcTasks
Assignement of IfcObjects ( e.g building elements) to IfcCostItems
You are welcome to start testing the above features with us.
And because Blender is, hands down, the most powerful 3D Software package in the world : we can start doing very very sweet construction animations.
** Coming features (Non-exhaustive):**
- Authoring of IfcConstructionResources,
- Bill of Quantities based on Quantities,
- Bill of Quantities based on Processes,
- Importing IfcWorkSchedule and IfcResources from proprietary project management software exports,
- Importing IfcCostSchedule from external Schedules of Rates,
- Automated Animation options in Blender.
.... Explore Opportunity and Risk Analaysis Workflows, from the vast and intertwined data at our disposal.
@brunopostle said:
I thought "that's crazy, having pricing information inside a design file", but maybe this is what architects need: a big dollar value in the corner of the screen that keeps going up while they are designing, it could be animated with gold coins and a ka-ching noise ;)
@Moult@SigmaDimensions I get this error while trying to add a cost schedule. I am suspecting it's because the .ifc file is was authored using IFC2x3 schema. Am I right, or there could be other reasons for the error?
@iosvarms - you are correct. Cost schedules, Work Schedule and resources are being implemented in IFC4 only for the moment.
@Moult , given the updates and improvements brought to IFC4, I feel like working on IFC 2x3 may limit many features - what do you think should be done?
@SigmaDimensions agreed, it is not a high priority for me to support IFC2X3 right now for cost / sequencing, and it feels as though we'd be building something for a schema that is missing functionality anyway. @iosvarms you can try out the "Migrate" IfcPatch recipe and use ["IFC4"] as arguments, and it can convert your IFC2X3 to an IFC4 file for you.
There was a recent bug where during the conversion it would spit out gigabytes of data which I recently fixed, so uh... be careful if you try it :)
Hello Everyone, We are now resuming the Weekly hacking Sessions on costing @ 10 AM UTC starting the 15/05/21.
We are getting to a point where we can start doing a few demos regarding the new features @Moult and I have been working on with regards to Construction Planning, Cost Scheduling, and Resources Management.
@iosvarms@dav80@Jesusbill@walpa@Aldo, If you'd like to join us on the Saturday sessions - this will be our first opportunity to get feedback in a live session, but also take the required time to explain things in detail for these modules.
@Aldo there is a difference between what you have created and the buildingSMART example. In their example, they have used components to add together things, whereas you have simply added more cost values. However, since you are "adding", the result is the same. Another difference is that you have added 2 category cost values (Labour + Material) which is equivalent to their sum cost value (CostType=*). Again, they are equivalent (but theirs is more concise), so you get the same result.
@Jesusbill thanks! I notice the video is unlisted. What are people's thoughts on whether these are listed, and how they should be advertised? Should they belong on the OSArchOrg channel, or separate?
Hi guys,
I am researching on model-based costing and came by accident over this initiative. Today I watched the recording. I was impressed by the possibilities and would like to join you at the next session. I was wondering when is it scheduled? This saturday, may 22nd, or next, may 29th?
@Moult said:
[...] What are people's thoughts on whether these are listed, and how they should be advertised? Should they belong on the OSArchOrg channel, or separate?
As discussed in the live chat, I say let's host these recorded WIP demos under a new playlist in OSArch account for archiving / centralisation, but also through the content creator's youtube account. We should be seeing more of these Technical Sessions/ Demos on blenderBIM modules & Open Source workflows in the near future!
@Dafu99 , great accident - the internet is wonderful!
You are more than welcome to join the next Demo session, the 29th of May @10AM UTC should do - although it had not yet been announced (need to build a few more features to make it worthwhile :) ) .
We can certainly give more time to questions and feedback on the 29th, after you guys have tried the features demonstrated in the live recording above.
By the way, for everybody trying out the new features, especially those who are professional cost planners and program schedulers, please report bugs and submit ideas for improvements, no matter how trivial it may seem! Do not assume that it will be fixed without your valuable input - I am only trained in architecture and have zero domain knowledge of what is important or useful to your domain :) You can either post feedback here on this thread, or on the Github issue tracker.
I highly recommend starting a brand new project and attempting to do it 100% in the BlenderBIM Add-on with zero proprietary software. In approximately 15 minutes of attempting to do so, you will find that the BlenderBIM Add-on is missing a vital feature - let us know, so we can fix it :)
We're about to dive into IfcResources implementation into more detail, and a few points regarding the specifications need to be raised: Attached is my shot at making a Logic Map for Implementation of cost / planning / resources to make it easier.
The diagram ( and comments) focus more on IfcResources, an how it ties to Processes and to Cost items - Let's discuss these @walpa@Moult ?
Regarding Costing and Work Planning: a few questions and clarifications have been raised in the BuildingSmart forums by @Moult for clarifications/questions/corrections etc. - We have yet to do the same for IfcResources and related concepts, after this discussion.
I will post here a couple of considerations about resources and costing. One thing I'd expect from a costing software would be to be able to create different type of workers/labour with a different cost per hour and assign a them to an IfcCostItem in different quantities plus a product. An example below.
Low Voltage cable type XYZ, with a cross section 3x1,5 mm2 (unit measure: 1m), should be composed by the following elementary costs:
Low Voltage cable type XYZ, with a cross section 3x1,5 mm2 : 1m
Expert (*) Electrical worker: 0,04h
Apprentice () Electrical Workers: 0.04h
() depending on the different country laws these definitions will change/vary.
From what I have seen in @SigmaDimensions diagram this should be possible, but I thought it could be useful to add a user case.
Another thing that I think will be useful is a script to compile IfcQuantityLength for some elements, for example cable trays quantities may be counted by using their weight (rarely in my personal experience) or commonly using their linear length. Volume or surface are not normally used.
I tried to replicate the demonstrated features but seem to lack some functionality. For example, I cannot add cost items to the cost schedule when hitting the pencil button; this only creates a second cost schedule without the possibility to edit:
Another problem was encountered when converting a 2x3 file to IFC4. Blender closed down and the produced IFC was corrupt missing two lines at the end:
Probably an initial error message at startup has something to do with these issues:
I downloaded the latest built (0.0.210405), I guess this version was used in the demo?
Have I missed out on a basic setting?
I have attached the converted IFC4 file if anyone wants to test attaching cost items (I have manually inserted the two missing lines, so the file should comply to the IFC schema).
I hope someone can help me on the way ;-)
Hey @Dafu99 - The blenderBIM version you are using did not yet contain the features demonstrated, we were using the latest developer testing build in the video.
You can download the latest developer testing build here; The latest build is often more stable and has more features - For blender versions =< 2.92, you need the py37 version.
I'll be posting a few samples files with costing/scheduling data, to help with the demo above.
I'm not certain why the conversion IFC 2x3 to IFC4 didn't work. @Moult can probably shed some light on this
@dav80 you can choose which quantity (length vs weight, etc) when linking elements to the cost item.
@Dafu99 yeah download the latest developer testing build and try again. The version from the website is old :) Also try the conversion from 2X3 to IFC4 again with the latest build. If it still doesn't work, send me the IFC2X3 file so I can try to fix the migrator script. Use dion@thinkmoult.com if the file is private.
@SigmaDimensions and @Moult with the latest built both the conversion and adding cost items works nicely! However, I cannot edit the cost items once created (the pencil item is missing) - a bug?
@Dafu99 - that's odd. Did you get any errors before it stoped showing?
The "editing pencil button" should show for the values of that Cost Item. Does it also happen for the cost values of the Cost Item called " Test" above it? Mind sending me your file so I can try it out?
@SigmaDimensions the bug is likely the active cost {value/quantity} id is still active on another cost item. We should have it reset to 0 whenever another cost item is changed. Do you mind fixing this?
@Dafu99 if my theory above is correct, you can just perhaps click around to find the last cost value you were editing (which will likely be in another cost item), and press the "disable editing" or finish editing button.
@dav80 sorry for not getting back in detail to your question. @SigmaDimensions and I went through the graphs today and the conclusion so far is that it's really confusing - but we are slowly figuring it out. @SigmaDimensions has a set of conclusions he has drawn, and I am still processing what he has figured out already to "reconfirm" his conclusions. Hopefully he'll describe how your scenario can work.
It is still a bit too soon for the next costing/scheduling demo, so tomorrow's ( Satuday the 29th) WIP demo is cancelled. @Dafu99, @dav80, @iosvarms , @Aldo , @walpa, @Jesusbill, @Moult ... Please accept my apologies if this has messed up your plans.
I'm working on presenting this Video Demo In a Wiki Page - containing a small guide for Costing and Scheduling + sample files ( Haven't yet promoted the video so that more users can try this out, and need to do so..)
@Dafu99 said: @SigmaDimensions and @Moult with the latest built both the conversion and adding cost items works nicely! However, I cannot edit the cost items once created (the pencil item is missing) - a bug?
Fixed - the latest build doesn't have this issue anymore.
A little update that there is now code that considers the entire project and calculates critical path, float, early/late starts, and early/late finishes. It works for all relationships (FS, FF, SF, SS). In addition, the gantt chart has been updated to visualise this, along with a fixed header (for larger projects).
A word of warning that in order to calculate the critical path, a full analysis of the task network is required. This is exponentially expensive to compute, so whilst it computes almost instantaneously for a tiny test of under 50 tasks, for a medium project of 500 tasks, I got bored after waiting for 15 minutes and force quit - after it had calculated over 2 million path combinations. For a large project of 2,000 tasks, this is probably insane.
There are a few methods to get over this:
Maybe there is a more efficient algorithm for calculating critical paths and forward / backwards pass for float and start / finish calcs. I don't know what it is, and my brain isn't built for this, but maybe yours is? Reach out!
Maybe there is a way to make this multicore. Perhaps, but again, not my strength.
Maybe Python isn't the right approach.
Life is hard, it takes that long to compute and that's normal - that's OK, for the majority of projects, I suspect we'll only be doing "incremental computing" i.e. recalculate only a subset of the graph whenever a change is made. The downside is that if you're importing a large dataset and that format doesn't store this data, we can't recalculate it.
@brunopostle indeed I was using networkx, and using their in-built algorithms.
The good news is that the code has been now rewritten and optimised, thanks to @SigmaDimensions who pointed out another video showing another method of doing the calcs by hand (which is much simpler than the method I saw taught in another YouTube tutorial! That's why we need domain experts!). This method is much faster which is O(n²) in the worst case, where n is the number of tasks. With 500 tasks, it is now calculated in under a second, so as far as I'm concerned, this is a huge win. In theory, the new approach may not even require networkx since we no longer use any of their algorithms, but there is still a bit of work to do before this is finished, so we'll wait and see.
Things still to do:
Take into account different calendars during float and early / late date calculations and resolve durations into dates
Calculate free float
Do an incremental calculation (i.e. if you only change one task)
Comments
Update on Project Management features - Construction Scheduling and Cost Scheduling.
After 2 weeks of work with @Moult , we are really excited to annouce that we have now implemented in blenderBIM and the ifcopenshell.api, the following features:
You are welcome to start testing the above features with us.
And because Blender is, hands down, the most powerful 3D Software package in the world : we can start doing very very sweet construction animations.
** Coming features (Non-exhaustive):**
- Authoring of IfcConstructionResources,
- Bill of Quantities based on Quantities,
- Bill of Quantities based on Processes,
- Importing IfcWorkSchedule and IfcResources from proprietary project management software exports,
- Importing IfcCostSchedule from external Schedules of Rates,
- Automated Animation options in Blender.
.... Explore Opportunity and Risk Analaysis Workflows, from the vast and intertwined data at our disposal.
Who's excited?
The least of features is impressive! Looking forward to it!
We strive to listen to users. Step 1: https://github.com/IfcOpenShell/IfcOpenShell/commit/57396e1bb993c690d26c64b861a09bce6787bf23
@Moult @SigmaDimensions I get this error while trying to add a cost schedule. I am suspecting it's because the .ifc file is was authored using IFC2x3 schema. Am I right, or there could be other reasons for the error?
@iosvarms - you are correct. Cost schedules, Work Schedule and resources are being implemented in IFC4 only for the moment.
@Moult , given the updates and improvements brought to IFC4, I feel like working on IFC 2x3 may limit many features - what do you think should be done?
@SigmaDimensions agreed, it is not a high priority for me to support IFC2X3 right now for cost / sequencing, and it feels as though we'd be building something for a schema that is missing functionality anyway. @iosvarms you can try out the "Migrate" IfcPatch recipe and use
["IFC4"]as arguments, and it can convert your IFC2X3 to an IFC4 file for you.There was a recent bug where during the conversion it would spit out gigabytes of data which I recently fixed, so uh... be careful if you try it :)
Hello Everyone,
We are now resuming the Weekly hacking Sessions on costing @ 10 AM UTC starting the 15/05/21.
We are getting to a point where we can start doing a few demos regarding the new features @Moult and I have been working on with regards to Construction Planning, Cost Scheduling, and Resources Management.
@iosvarms @dav80 @Jesusbill @walpa @Aldo, If you'd like to join us on the Saturday sessions - this will be our first opportunity to get feedback in a live session, but also take the required time to explain things in detail for these modules.
Everyone who is interested can join us.
Bumping this thread, as we are meeting tomorrow! @SigmaDimensions can you confirm the location where we're meeting, and whether this will be recorded?
For those joining us tomorrow tomorrow the 15/05/21 for demos on costing ( and a bit of scheduling), @ 10 AM UTC :
[Edit]: This link should lead you to the meeting room, https://demo.bigbluebutton.org/gl/yas-kxk-hdl-ojm
Please do check the Live chat if you have any issues joining
Hello, I was trying to reproduce the example from last saturday. Is it correct?
Hi all, here is the video recording from the tech session:
@Aldo there is a difference between what you have created and the buildingSMART example. In their example, they have used components to add together things, whereas you have simply added more cost values. However, since you are "adding", the result is the same. Another difference is that you have added 2 category cost values (Labour + Material) which is equivalent to their sum cost value (CostType=*). Again, they are equivalent (but theirs is more concise), so you get the same result.
@Jesusbill thanks! I notice the video is unlisted. What are people's thoughts on whether these are listed, and how they should be advertised? Should they belong on the OSArchOrg channel, or separate?
amazing
Hi guys,
I am researching on model-based costing and came by accident over this initiative. Today I watched the recording. I was impressed by the possibilities and would like to join you at the next session. I was wondering when is it scheduled? This saturday, may 22nd, or next, may 29th?
As discussed in the live chat, I say let's host these recorded WIP demos under a new playlist in OSArch account for archiving / centralisation, but also through the content creator's youtube account. We should be seeing more of these Technical Sessions/ Demos on blenderBIM modules & Open Source workflows in the near future!
@Dafu99 , great accident - the internet is wonderful!
You are more than welcome to join the next Demo session, the 29th of May @10AM UTC should do - although it had not yet been announced (need to build a few more features to make it worthwhile :) ) .
We can certainly give more time to questions and feedback on the 29th, after you guys have tried the features demonstrated in the live recording above.
By the way, for everybody trying out the new features, especially those who are professional cost planners and program schedulers, please report bugs and submit ideas for improvements, no matter how trivial it may seem! Do not assume that it will be fixed without your valuable input - I am only trained in architecture and have zero domain knowledge of what is important or useful to your domain :) You can either post feedback here on this thread, or on the Github issue tracker.
I highly recommend starting a brand new project and attempting to do it 100% in the BlenderBIM Add-on with zero proprietary software. In approximately 15 minutes of attempting to do so, you will find that the BlenderBIM Add-on is missing a vital feature - let us know, so we can fix it :)
We're about to dive into IfcResources implementation into more detail, and a few points regarding the specifications need to be raised: Attached is my shot at making a Logic Map for Implementation of cost / planning / resources to make it easier.
The diagram ( and comments) focus more on IfcResources, an how it ties to Processes and to Cost items - Let's discuss these @walpa @Moult ?
Regarding Costing and Work Planning: a few questions and clarifications have been raised in the BuildingSmart forums by @Moult for clarifications/questions/corrections etc. - We have yet to do the same for IfcResources and related concepts, after this discussion.
I will post here a couple of considerations about resources and costing. One thing I'd expect from a costing software would be to be able to create different type of workers/labour with a different cost per hour and assign a them to an IfcCostItem in different quantities plus a product. An example below.
Low Voltage cable type XYZ, with a cross section 3x1,5 mm2 (unit measure: 1m), should be composed by the following elementary costs:
() depending on the different country laws these definitions will change/vary.
From what I have seen in @SigmaDimensions diagram this should be possible, but I thought it could be useful to add a user case.
Another thing that I think will be useful is a script to compile IfcQuantityLength for some elements, for example cable trays quantities may be counted by using their weight (rarely in my personal experience) or commonly using their linear length. Volume or surface are not normally used.
I tried to replicate the demonstrated features but seem to lack some functionality. For example, I cannot add cost items to the cost schedule when hitting the pencil button; this only creates a second cost schedule without the possibility to edit:
Another problem was encountered when converting a 2x3 file to IFC4. Blender closed down and the produced IFC was corrupt missing two lines at the end:
Probably an initial error message at startup has something to do with these issues:
I downloaded the latest built (0.0.210405), I guess this version was used in the demo?
Have I missed out on a basic setting?
I have attached the converted IFC4 file if anyone wants to test attaching cost items (I have manually inserted the two missing lines, so the file should comply to the IFC schema).
I hope someone can help me on the way ;-)
Hey @Dafu99 - The blenderBIM version you are using did not yet contain the features demonstrated, we were using the latest developer testing build in the video.
You can download the latest developer testing build here; The latest build is often more stable and has more features - For blender versions =< 2.92, you need the py37 version.
I'll be posting a few samples files with costing/scheduling data, to help with the demo above.
I'm not certain why the conversion IFC 2x3 to IFC4 didn't work. @Moult can probably shed some light on this
@dav80 you can choose which quantity (length vs weight, etc) when linking elements to the cost item.
@Dafu99 yeah download the latest developer testing build and try again. The version from the website is old :) Also try the conversion from 2X3 to IFC4 again with the latest build. If it still doesn't work, send me the IFC2X3 file so I can try to fix the migrator script. Use dion@thinkmoult.com if the file is private.
@SigmaDimensions and @Moult with the latest built both the conversion and adding cost items works nicely! However, I cannot edit the cost items once created (the pencil item is missing) - a bug?
@Dafu99 - that's odd. Did you get any errors before it stoped showing?
The "editing pencil button" should show for the values of that Cost Item. Does it also happen for the cost values of the Cost Item called " Test" above it? Mind sending me your file so I can try it out?
@SigmaDimensions the bug is likely the active cost {value/quantity} id is still active on another cost item. We should have it reset to 0 whenever another cost item is changed. Do you mind fixing this?
@Dafu99 if my theory above is correct, you can just perhaps click around to find the last cost value you were editing (which will likely be in another cost item), and press the "disable editing" or finish editing button.
@dav80 sorry for not getting back in detail to your question. @SigmaDimensions and I went through the graphs today and the conclusion so far is that it's really confusing - but we are slowly figuring it out. @SigmaDimensions has a set of conclusions he has drawn, and I am still processing what he has figured out already to "reconfirm" his conclusions. Hopefully he'll describe how your scenario can work.
It is still a bit too soon for the next costing/scheduling demo, so tomorrow's ( Satuday the 29th) WIP demo is cancelled.
@Dafu99, @dav80, @iosvarms , @Aldo , @walpa, @Jesusbill, @Moult ... Please accept my apologies if this has messed up your plans.
I'm working on presenting this Video Demo In a Wiki Page - containing a small guide for Costing and Scheduling + sample files ( Haven't yet promoted the video so that more users can try this out, and need to do so..)
Fixed - the latest build doesn't have this issue anymore.
@SigmaDimensions - works like a charm now. Thanks for the fast update!
A little update that there is now code that considers the entire project and calculates critical path, float, early/late starts, and early/late finishes. It works for all relationships (FS, FF, SF, SS). In addition, the gantt chart has been updated to visualise this, along with a fixed header (for larger projects).
A word of warning that in order to calculate the critical path, a full analysis of the task network is required. This is exponentially expensive to compute, so whilst it computes almost instantaneously for a tiny test of under 50 tasks, for a medium project of 500 tasks, I got bored after waiting for 15 minutes and force quit - after it had calculated over 2 million path combinations. For a large project of 2,000 tasks, this is probably insane.
There are a few methods to get over this:
Relevant commits:
- https://github.com/IfcOpenShell/IfcOpenShell/commit/67b636f5fb9bcc0a821fb25e1e3eb6b9c17395b1
- https://github.com/IfcOpenShell/IfcOpenShell/commit/8338cc72dbb620a2562d09956518c4e440a36353
Screenshot. Red bars are tasks in the critical path.
@Moult Are you using a graph library? There are algorithms like dijkstra that speed these things up
@brunopostle indeed I was using networkx, and using their in-built algorithms.
The good news is that the code has been now rewritten and optimised, thanks to @SigmaDimensions who pointed out another video showing another method of doing the calcs by hand (which is much simpler than the method I saw taught in another YouTube tutorial! That's why we need domain experts!). This method is much faster which is O(n²) in the worst case, where n is the number of tasks. With 500 tasks, it is now calculated in under a second, so as far as I'm concerned, this is a huge win. In theory, the new approach may not even require networkx since we no longer use any of their algorithms, but there is still a bit of work to do before this is finished, so we'll wait and see.
Things still to do: