OpenDataBIM - creating open and simple data
Hi community,
we are a team of developers who develop what we call a BIMjson (JSON + DAE) format and free applications to work with this format.
Our website:
https://opendatabim.io/
GitHub:
https://github.com/OpenDataBIM
YouTube:
https://www.youtube.com/channel/UCQ3IWLMLnBFwNeG5VBxXQ5Q
The goal of the project is originally to create an industry-wide data converter so that any professional with a closed RVT or parametric IFC could simply and quickly translate their data into a simple textual open format. In order to make this converter free someday, we need the support of a community that still needs to be built. So now we have a business and Enterprise app that lets you convert RVT and IFC formats to a single, open-text format.
For individual professionals, we are developing a Web converter, which will be free for files of a certain size (here we do not know the economics of conversion through server capacity yet).
We will be glad to any questions
Comments
Hi Artem, great to see you here.
So a lot of your scripts are under FOSS licenses, and is the conversions to BIMjson a proprietary service (free or not)? Also when you say that BIMjson is an open format where can I see some documentation of the format itself? I see there are plenty of examples on github but is there documentation of the conversion process? I suppose a format can indeed be open without much documentation, but it can't gain traction without consistent implementation.
For some of this conversion from IFC to other formats IfcOpenShell or maybe Xeokit might have already done the work if you want to collaborate with them.
Hey @OpenDataBIM ! Cool stuff! Is it JSON+DAE or CSV+DAE? The apps you're building seem to expect CSV. As @duncan says, it would be great to see some specs about the format and whether there is a FOSS-licensed converter that we can run to try out your apps.
Did you know there is an official IFC-JSON serialisation already? This might save you a lot of work for converting IFC to JSON since it already exists. Similarly for CSV, there are a number of existing IFC bulk editing and scheduling tools that use CSV, both free and proprietary.
I see you offer converting RVT to plaintext. Having free software for this task would be awesome! It would certainly give ODA something to think about. What's the process for this? Where can I find the code?
Thank you for your questions, the original idea was to give a converter for the whole industry, so that any professional could quickly and easily translate their data into a simple open format.
In the beginning of the project, we went to all the major specialists in central Europe and proposed creating a non-profit organization, something like "Doctors Without Borders", but at the time no one was interested, everyone was happy with IFC or Forge. No one saw the point in competing with IFC. So far the documentation on the conversion is only visible to our customers, unfortunately, to support the development of the converter and keep up with changes from Autodesk or buildingSMART every year we need to sell solutions to businesses so that we can continue to fund the development of free solutions.
Now we're selling part of our solution to businesses in order to further drive the development of the format itself and free applications to it. JSON+DAE = XML+DAE = CSV+DAE = XLS+DAE. Data on properties and parameters for the project are saved as a list of elements with a unique ID, for which there is also a geometry in the DAE file. So geometry and properties are linked by ID. If we only need properties, we can work with JSON, XML, CSV, XLS files, if we need also geometry we can load it from DAE format.
We do not have a goal to replace any tools, we are trying to supplement the tool palette with a simple converter that will allow you to quickly get clean data, we don't see the need to create interaction with IfcOpenShell yet, since we already get the information in a simple and open form with our tools and it is easier to work with Jupyter Notebook than with ifcOpenShell. About IFC-JSON and IFC-CSV - it would be interesting to compare the quality and simplicity of the data. If you have examples of IFC that you converted to JSON and CSV please send them here, and we can compare the data we get.
Of course it's not easy to form a new community. But there are a couple of ideas that we don't want to disclose yet that might lead us to free tools and independence from organizations like buildingSMART and Autodesk
There are two interesting parts to this discussion. One is technological implementation of interesting ideas - the second part is how the project fits into the ecosystem of libre/free software, open source software & open formats.
At the moment since this forum is for Open Source in Architecture and the topic is "creating **open **and simple data" I'd like to focus on that second aspect.
Maybe a separate thread can be started to discuss the technology, I'm sure some people here would be happy to throw ideas around.
@OpenDataBIM , @Moult & I got started with a bunch of questions, but the ones about FOSS are the most important here. Maybe you could focus on answering those questions? OSArch is a great place to talk about and promote libre/free software - but we're strict in our definitions of what that means, and we keep a sharp eye out for open-washing. So let's avoid going down that path by being very precise - which parts of your project are currently libre/free software, open source software & open formats. Just to be clear free (gratis) tools are not what we're interested in, we're interested in free (libre) solutions.
Here are some links that might be useful if there are terms we're unclear about OSArch definition of Free Software, Free Software Foundation definition of Free Software, IBM has this quite good article about Open standards vs. open source: A basic explanation
Agree that before we look at the technology, our responsibility here at OSArch is to promote open source in AECO.
From your description, it sounds as though you provide a proprietary converter which converts both proprietary formats (i.e. RVT) and open formats (i.e. IFC) to a proprietary format (BIMjson). Note that even though you use CSV, XML, or JSON, it doesn't mean that your format is open. For example, Microsoft Project XML is proprietary. Then, after the proprietary converter into a proprietary format, you provide some open source scripts to visualise or manipulate this data.
As noted in our wiki there are 4 broad combinations of proprietary and open source:
It sounds like your offering falls into the second category (for the converter) and the third category (for the apps after the conversion) - free software that depends on proprietary software. OSArch has a clear focus on those in the first category, but those in the latter 3 categories are a bit of a grey area. We have content on the Wiki covering the 4th category, with an aim to help people transition over to free software. We also have one page about the 3rd category, but nothing for the 2nd category.
I'd like to hear from others in OSArch what they think about this. I personally feel that the title OpenDataBIM is misleading. If anything, it is the opposite: converting Open Data (IFC) into proprietary data (BIMjson).
Would you consider opening up your work to be fully open source? It is a great adventure, and it is not mutually exclusive with providing commercial services. Alternatively, would you consider contributing to existing open source efforts as a foundation? That way the improvements you make help the rest of the community.
If it is not a free (libre) project this thread is more like SPAM...
Why can't the people be clear and transparent about their intentions? Is it too difficult to say these are my intentions and this is the license?
Ambiguous intentions from a proprietary project in a open community cannot be a good signal.
PD: OSArch should have a message for those who want to publish proprietary projects promotions, explaining that this is not the place for them.
@bitacovir this thread is not spam. I invited @OpenDataBIM ( Artem ) here to tall us about this project since he promotes it as open and parts of it use FOSS licenses. So be a bit more polite please. Also, we have everything to gain from any dialogue with any software project. We can explain our aims over and over again and more people will hear them.
Let Artem explain his project clearly with reference to our questions and then we can discuss if OSArch has anything to offer. At the very least we can showcase the permissively licensed projects he is very welcome to build upon.
I'm not here to protect anyone from fair criticism, but everyone deserves time to reflect on their answers in a respectful atmosphere.
Yes, now our converter is a proprietary product and this belongs to the "gray zone" according to your classification, now because of the involvement of paid resources it is not possible to make the converter fully open. BIMjson format is a digital data storage specification that is license-free in use. We will prepare the specification in such a way that we can share it with the community. "personally feel that the title OpenDataBIM is misleading" - we convert complex and proprietary data for companies that use our converter - into open and simple data. We did not write anywhere about the opensource converter, but only about the openness of the data that we get with this converter and those open tools that can replace the functionality of proprietary products. buildingSMART is also not an open structure developing standards: buildingSMART and the structure of the IFC is determined today by lobbying groups.
For us, this style of communication is an outdated format of communication, as we have often pointed out.
For example, in Russia we have hundreds of groups with thousands of specialists who are ready to openly discuss any topics related to BIM, also everything related to OpenSource:
https://miletoda.notion.site/miletoda/1a54f99e8ca14de29e0d6d07ef40ae03?v=a85e1dc271764d208091c5ff8a851815
On our side, if you want to learn about BIM and OpenSource technologies from the other side of the world, you can also come to us. We also discuss OpenSource with leading software developers in Asia and Europe. All opinions are welcome
Commercial ventures and open source is not mutually exclusive. I hope you are not under the impression that if you make something open source, you can't sell it. Would you consider changing your approach? You might find that it actually increases your sales, especially as what you're selling seems to be the simplification of data - the audience would generally not be those to process data themselves.
In OSArch, our priority is on promoting Open Source software. As you've seen, there are differing opinions once you cross into the grey zone. I would encourage considering opening up things further, otherwise you may find a lack of support from some in the community depending on their alignment. For example, a good start would be publishing data specifications under an open source license. You may also consider dual licensing. Or selling support services on top of an open source core.
I have been vocal in the past about buildingSMART's "closed door" impression of developing IFC. However, that has gradually changed over the past years since their forums were launched. See for yourself to get involved: https://github.com/buildingSMART/IFC4.3.x-development . Now, your statement is definitely not true. A number of us have contributed to the specification, and we are individuals, not lobbying groups. I have personally been on the watch for ways for individuals and the wider public to contribute, and I have not seen you attempt to get involved, so it sounds like you have an outdated impression.
The link you post seems to contain two channels related to open source (the majority seem to be proprietary software). I don't read Russian unfortunately. Which software developers are you working with? Which open source projects are they part of?
Yes, of course, in Russia we have a community and we promote there the principle of open source. Unfortunately, there are no free grants or funding in Russia, which would allow people to understand how important OSS is. It is a hard process, but it seems we are able to convey to "our specs" the importance of collaborative development based on open source principles.
This is a long topic, sorry we started it here. We are talking to Russian software developers on our side, as well as people who were behind the creation of the IFC format itself in the 90s. From them and from those who today promotes OpenBIM and BuildingSMART at the ministerial level in some countries, we know that this is buildingSMART and the structure of the IFC is determined today by lobbying groups.
Not to go on about this: we agree with you that the IFC and buildingSMART are fighting for OpenSource.
Which software developers are you working with?
In our chats you can find most of the developers behind BIM products: from people who develop Revit or Autocad to developers who are behind NanoCAD, OpenCascade, OnShape, ODA, Renga or Revizto, as well as other Russian speaking specialists working both in Russia and abroad. You can find any specialist in any subject. Specifically we are developing 5D BIM, which in our opinion is one of the most complex, but at the same time the most important in the construction industry. As soon as we have a working solution for the 5d cycle, we will open the code for it and try to connect to the global community. While we are still at the beginning of the journey.
Now Duncan is in one of our chats, where news of our project is published and, in general, topics related to open source tools are sometimes discussed. Among the participants there are representatives of buildingSMART and representatives of some software teams
https://t.me/opensourcebim
It's hard to communicate through email notifications and through listing questions and answers in one post, come chat if you're interested in discussing something.
Here the main questions relate to our name and the transparency of the data that we give. We convert complex and proprietary data for companies that use our converter - into open and simple data, we are talking about the openness of the data that we get with this converter and those open tools that can replace the functionality of proprietary products. We have just started to develop and today on a certain subscription we give full access to the code.
I have great respect for the members of the OSArch community that you lead. Therefore, I would like to ask a question first to your community. OSArch community today is mostly focused on building infrastructure around the IFC format (OpenBIM movement).
Dion, please tell us where the name openBIM came from and who first started selling the openBIM abbreviation. If you have the opportunity and you know, you can also tell how buildingSMART appeared and the IFC format itself, on which your community builds most of its products.
@OpenDataBIM the concern I have over the name is quite simple. The name OpenDataBIM suggests "open data" and "open BIM". Upon closer inspection, it seems to be neither.
It is not Open Data (see definition) because someone paying you to convert their data from one format to another does not change the copyright ownership of the data. The data does not magically become free to republish. If you want to see examples of Open Data, look at the OSArch Wiki Open Data Directory.
There are a few definitions of OpenBIM out there. You can read more in the OSArch Wiki page on OpenBIM. What is common to all definitions is that BIM data is defined using an Open Standard (Wikipedia definition). This distinguishes between a random person creating a random CSV file and calling it OpenBIM and people exchanging CSVs using an agreed and published standard. By this definition, the IFC schema is one of many OpenBIM standards. You can see a list of Open Data Standards in the OSArch Wiki. However, because I cannot find a publicly available standard for BIMJson, it is unlikely to be considered OpenBIM unless you are very generous with your definition.
OSArch is focused on building and promoting open source software. Our first priority is open source. Sometimes, this involves open data standards, like IFC, BCF, Brickschema, BHoM, and so on. IFC and BCF is publicly available and published under under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0). (Note NC-ND-clause makes it a non-free license, however the definition of an open standard is less black and white). It also happens to cover a wide variety of domains in BIM. This makes it naturally something that turns up again and again. IFC is not the only focus in OSArch. For example, Brickschema is free and open-sourced under the BSD 3-Clause license. CityJSON is under the public domain, and the Open Web Foundation Agreement Version 1.0. BHoM uses the LGPL-3.0 open source license. When two open source software integrates, we need to be transparent about how the data schema works - so open data specifications play a natural role.
OSArch is collectively led. The concepts and definitions of open source and free software are well established in the software industry. OpenCAx developments are also not a new field, though BIM authoring is relatively new. So you will find many community members in OSArch who feel very strongly about people misusing or "openwashing" in AEC, especially as the wider AEC industry may not know how to differentiate open as it is recognised in the software world. For example, many people confuse "Open API" with "Open Source". Another example is that some may be misled into thinking Unreal Engine to be open source, which is not correct. OSArch can help ensure that people understand the freedoms they deserve from software, and protect them from being misled from vendors who market themselves as open, but freedoms are not provided.
A better description of what you are doing is "converting complex and proprietary data for companies that use our converter into simple plaintext data". Simple plaintext data is a step in the right direction - plaintext data is easier to interoperate and reverse engineer. However, something being plaintext (JSON/XML/CSV) does not mean the conversion is open source (your converter is proprietary), it does not mean the data is open data (the data remains copyrighted), and it also does not mean the data is based on open standards (your format has no publicly available specification). It is no different right now to someone pressing "Export as P6 XML" in the proprietary software "Oracle P6".
There is one aspect of your offering that is open source - the scripts at the end which visualise the data using your proprietary BIMJson format. This is a fantastic start - and we'd love to encourage you to take the next step! If you'd like to help OSArch in its journey to promote open source, please consider relicensing your converter under an open source license. As for the data specification, please consider publishing a publicly available open standard for it, such as using Creative Commons. Another great initiative would be to publish open data datasets for sample buildings for people to experiment with.
It's just mega inconvenient to communicate in large chunks of text with answers and questions at the same time. This kind of forum format is outdated. Most of these questions were answered in the chat room:
https://t.me/opensourcebim
This is real communication, with short questions and short answers. Please, if you get a chance, you can join in to discuss these questions and then leave the chat room again. Start discussing and answering questions:
https://t.me/opensourcebim/752
About the specification: we wrote in the previous messages - we are doing it. For the time being, it makes no sense for us to throw out a raw specification that has not been tested in many companies. You demand that in one month we reach the level of openness to which IFC, originally controlled by Autodesk, and over 30 years of its existence, has finally grown to the buildingSMART lobbyist organization. Now, in our list of priorities - it was not important. I hope you yourself understand that in order to create open products you need to receive funding. We first need to resolve the issue of where to get funding for the development of open source tools. Additionally, at the moment we are leaning towards the creation of a decentralized organization - like a DAO to create platforms that can help the construction industry become more transparent.
What also seems wrong to me is to develop products under IFC, which is probably also the fruit of the lobbyist groups of certain vendors. What you don't want to say: the openBIM movement was started by several CAD vendors in central Europe, they are closed proprietary solutions that earn points by using the IFC format and promoting it in the European market. Maybe you and your colleagues also have double standards. Where it is beneficial, you can turn a blind eye to the lobby of proprietary vendors. You are not opposed to this, no one writes articles about it, although it lies on the surface. Proprietary software houses of central Europe, earning billions on CAD product - have done for you the certification of formats and standards, and you consider this a victory for the Opensource community. I have already met many people over the past year who sit in chapters and rums bs, and the people who govern bs. I can see their fatigue with the internal bureaucracy and the fact that the organization has gotten so big that it will now just spend more resources on keeping the organization alive than on real development.
We have our own view of platform development, and it's better if we don't look at what they're doing. They are great guys, but no one stops us from developing in parallel so that we don't influence each other too much. That's the point of OpenSource - we shouldn't merge, we should make new forks and branches.
I'd like to add some new flavour to the conversation by giving my take on OpenDataBIM.
First of all, I have to say I was initially intrigued by the solution that was being offered. Especially in the post-design phase, being able to extract useful information in a more user-friendly manner sounded like an easy win for me - In my eyes it was a simple case of 'the more the merrier', so I welcomed the fresh take. Unfortunately though, it didn't take me long to get a little discouraged. As @Moult also pointed out, the name "OpenDataBIM" is misleading; after looking at the website, searching google and searching the github page I couldn't find anything that allowed me to better understand the process of creating the advertised outputs. To me it was obvious that it indeed wasn't open source, which then brought me naturally to the following question:
To me it seems that the last thing our industry needs is yet another proprietary format.
I'm a relative newcomer to OSArch and OSS in general, so I'll happily excuse myself from getting into technicalities and definitions - In my humble opinion though, what makes OSS magical is the opportunity we have to tinker with software and data. The ability to study and analyse source code allows the community to Learn, Modify and Improve the software, all the while engaging with one another in a spirit of strong collaboration. I think it's fair to say that if OpenDataBIM isn't in fact open, the chances of it being promoted, used and discussed here are pretty slim.
I've read some of your articles on LinkedIn @ArtemBoiko and have found them genuinely interesting and fun to read. You seem like an intelligent professional with strong ideas on how to fix some of the issues in the industry. What bewilders me though is your zealous pursuit of discrediting the capability and value of IFC with constant reference to it's history (as you define it) and the so-called lobbyist groups. I've seen you go at this topic with @duncan in the telegram chat so I won't go into it again, but my overall feeling is that you're not doing yourself any favours with this rhetoric plus it makes it easier to discredit what you're saying because you appear to have a strong grudge. Even if everything you claim is to be true about its history, who cares? As Duncan said, IFC provides real value to the construction industry today and is an open standard with no barriers to users who want to contribute to and shape the standard for the future.
I do hope you stick around in this community - there are a lot of interesting discussions taking place and I think different voices come a long way to enriching the overall experience. Also, best of luck to any commercial ventures you and your team are undertaking, it's never easy and certainly the dilemma of opening up your IP to the public is never a black and white decision. Cheers, Vukas.
P.S. Regarding the forum format, both classical forums and chats have their pro's and cons. Blanket disregarding forums as old tech? I just don't understand why you would do that. Just the ability to logically sort discusssions, search for specific content and sheer readability of forums is enough to warrant their existence!
vpajic thank you for your message. We were not planning to come here ourselves and promote anything here. I have told Duncan many times on LinkedIn that we are not interested and we are not ready to cooperate with OSArch and the forum format itself does not seem appropriate in 2022, while we have great respect for the OSArch organization.
Duncan himself suggested registering on that platform and starting a thread discussing the project. Thanks so much for the invitation and for the opportunity to discuss, but it's really tiring to write entire articles here on questions with both questions and answers mixed in. In chat we correspond with short messages, which are convenient to send from any device. We chat there, usually in several chats at once.
It is not our goal to convince your openBIM community that it is run by lobbyists. These are all conspiracy theories that have nothing to do with reality. Just don't tell us how we should use our name if the openBIM community itself, using a double standard - forgetting that the very acronym openBIM was promoted by a corporation with billions of dollars in revenues, to create a competitive advantage.
You are right there is no difference in who created what. You can just turn a blind eye to it. But turning a blind eye to the current situation and management of buildingSMART is no longer possible. You have your own vision of the situation, and I have my own opinion on it, which was reinforced by communication with the main actors in all these stories about formats and organizations.
To create truly free products you need funding. No OpenSource project can be launched normally just in the evening after work, so to fight for OpenSource in today's world, you have to start making money first. I don't think we've done you much harm by taking the time to study the project. We didn't set out to mislead anyone, we're building interoperability and creating open data. We're building processes, not just trying to create some products for something we don't understand, which is why we started with the format. We're building our OpenSource and how we understand it.
Honestly, it's hard to communicate in the format of these emails. It's not modern communication. Come to chat rooms to exchange opinions, there the exchange of opinions is much more intense and faster. Especially if you're in the chat room, please write in the chat room, there will be more views of your message.
I have been following the above...
Perhaps it is my generation or my culture, but I am uncomfortable with the tone and some of the content above.
Call me a boomer but intelligent and polite long form conversation (I joined my first CAD forum in 1994) is not outdated and doesn't need to be replaced by 'chat room' short questions and short answers.
Also, I find all the references to 'lobbyist organization', 'Proprietary software houses of central Europe, earning billions on CAD product' and 'ministerial level in some countries' to be amusing and in this context ironic.
Dion (our leader, satirical) asked clear questions and provided references and explanations to what he asked, and I hoped to see clear answers so I could understand the service/project being discussed.
Instead, I detected a 'hard sell' through denigration. There is a subtly required for this type of communication that was sadly lacking.
I also detect defensiveness in relation to making money from selling the OpenDataBIM service, be proud of what you have achieved. We in the OSArch community are doing it differently but I for one admire an entrepreneurial enterprise when I see one, promote it as it is.
Nickel hello, thank you.
Here again about the double standard: above the user bitacovir wrote that the entire thread SCAM, without understanding how our team even appeared here. You liked his post, and now you write about "smart and polite long conversation".
I'm glad you find it all ironic. You can overlook how a few CAD big corporations in Europe, with the help of bureaucracy and their connections in chapters and rums, promote their interests.
All the questions were answered either here or in the chat room: we just started working, we are here by invitation, and for us the goal of OpenSource is very important, which is what we are trying to convey to our community in Russia. We are also for everything good and against everything bad.
We did not intend to advertise ourselves here - we are here at the invitation (which has been sent to us more than once) to discuss the project. You can delete this thread if you have administrative rights, we do not mind. Thank you all very much for the invitation and for this opportunity to discuss the project. We have great respect for your community.
@ArtemBoiko maybe you have other people in your project you might want to encourage to have a chat with us - since there are now two accounts in this this thread I don't even know who I am talking to. I've several times now while I sit at the start of my work day tried to write about what the problem is with this discussion, but I'm pretty sure it is futile. I don't think I'll be participating in this discussion any more. But if you have anyone in your project who is interested in learning something from us rather than lecturing us, please consider encourage them to join the conversation. That may sound disparaging but it's all I can think of to add some value to this conversation.
And please, look up the difference between spam and scam, such misrepresentations are extremely unproductive.
Yes thanks, unfortunately you do not really answer our questions in chat either - you only tell us what is interesting to you. Of course there are still some team members who actively discuss the project in the Russian-speaking segment, but for them English is not the priority language in the communication, some also prefer using chat.
If you have any questions come to our chat rooms, our doors are always open to you. We love OSArch and OpenSource, but please do not force us to love and respect buildingSMART and openBIM.
https://t.me/opensourcebim