SeeBridge meeting @ Technion
March, 2017
WP4 Review
Virtual Construction Lab
© 2017, Dr. Ling Ma
Requirement of the enrichment result
Produce feature-based enrichment rules
Algorithm for identification of model features and rule-based model processing
© 2017, Dr. Ling Ma
Mapping entities
Mapping properties
Mapping constraints
© 2017, Dr. Ling Ma
Rules:
Advantage:
Disadvantage:
Fact:
IF a & b & c -> ?
© 2017, Dr. Ling Ma
Features | F1 | F2 | F3 | F4 | F5 | F6 |
---|---|---|---|---|---|---|
Type A | y | y | x | y | n | n |
Type B | n | n | y | x | y | n |
Type C | y | y | n | y | x | n |
Type D | n | n | y | y | y | x |
Find the 'unique' string (skip the comparison with ambiguous 'x' )
y | n | y | x | n |
y | x | y | y | n |
Not unique
© 2017, Dr. Ling Ma
Physical contact ?
Orthogonality
Length comparison
Width comparison
Elevation comparison
...
Find the 'unique' string (skip the comparison with ambiguous 'x' )
y | n | y | x | n |
y | x | y | y | n |
Not unique
© 2017, Dr. Ling Ma
Sacks, R., L. Ma, R. Yosef, A. Borrmann, S. Daum. and U. Kattel, Enhancement of a Semantic Enrichment Tool for Building Information Modeling. Journal of Computing in Civil Engineering, under review.
© 2017, Dr. Ling Ma
Feature1 | Feature2 | Feature3 | Feature4 | Feature5 | |
Type A | n | y | X | n | y |
Type B | n | y | y | n | y |
Rules:
Fact:
Obj 1 | n | y | y | n | y |
Obj 2 | n | y | n | n | y |
PB>PA
PA>PB
© 2017, Dr. Ling Ma
Feature Matching
Longitudinal | Transverse | Vertical | Convex | ... | |
Primary Girder | 1 | -1 | -1 | -1 | |
Transverse Beam | -1 | 1 | -1 | 1 | |
... |
Primary Girder = (1,-1,-1,-1)
Longitudinal | Transverse | Vertical | Convex | |
Object 1 | -1 | 1 | -1 | 1 |
Object 2 | 1 | -1 | -1 | -1 |
Object 3 | 1 | -1 | -1 | 1 |
Object 1 = (-1,1,-1,1)
Primary Girder | Transverse Beam | |
Object 1 | 180 degree | 0 degree |
Object 2 | 0 degree | 180 degree |
Object 3 | 60 degree | 90 degree |
© 2017, Dr. Ling Ma
Pairwise Feature Matching
f1 | f2 | f3 | |
---|---|---|---|
E1,E1 | 1 | 1 | 1 |
E1,E2 | -1 | 0 | 0 |
E1,E3 | -1 | -1 | 0 |
E2,E1 | 0 | 0 | 0 |
E2,E2 | 1 | 1 | 0 |
E2,E3 | 0 | 0 | -1 |
E3,E1 | -1 | -1 | -1 |
E3,E2 | 0 | -1 | 0 |
E3,E3 | 0 | 0 | -1 |
f1 | f2 | f3 | |
---|---|---|---|
O1,O2 | 1 | 1 | 1 |
O1,O3 | -1 | 1 | -1 |
O1,O4 | -1 | -1 | 1 |
O1,O5 | 1 | 1 | 1 |
O2,O1 | 1 | 1 | 1 |
O2,O3 | -1 | -1 | 1 |
O2,O4 | 1 | 1 | 1 |
O2,O5 | 1 | 1 | 1 |
O3,O1 | 1 | 1 | 1 |
O3,O2 | -1 | -1 | 1 |
O3,O4 | -1 | -1 | 1 |
O3,O5 | 1 | 1 | -1 |
O4,O1 | 1 | 1 | -1 |
O4,O2 | 1 | 1 | -1 |
O4,O3 | -1 | -1 | -1 |
O4,O5 | -1 | -1 | 1 |
O5,O1 | -1 | 1 | -1 |
O5,O2 | -1 | 1 | -1 |
O5,O3 | -1 | 1 | -1 |
O5,O4 | 1 | -1 | 1 |
3 types of bridge elements (e.g., girder, sheer key etc.)
5 objects in the model
3 types of features
E1,E1 | E1,E2 | E1,E3 | E2,E1 | E2,E2 | E2,E3 | E3,E1 | E3,E2 | E3,E3 | |
---|---|---|---|---|---|---|---|---|---|
O1,O2 | 1 | 3 | -3 | 1 | -5 | 2 | -3 | 2 | 1 |
O1,O3 | -5 | 1 | -3 | 2 | -3 | -3 | 2 | 2 | -2 |
O1,O4 | -2 | 3 | -3 | 5 | 3 | -1 | -5 | -2 | -2 |
O1,O5 | -1 | 5 | 1 | 5 | 2 | 1 | 3 | 2 | 1 |
O2,O1 | -3 | 4 | 2 | 0 | 1 | -1 | -4 | 3 | -5 |
O2,O3 | -1 | 3 | 5 | -5 | 0 | 0 | -3 | 4 | -2 |
O2,O4 | 1 | 3 | -4 | 4 | 0 | 4 | -1 | 5 | -5 |
O2,O5 | -5 | -2 | -1 | 0 | 1 | 3 | 2 | 0 | 1 |
O3,O1 | -3 | 5 | 0 | -2 | -3 | 1 | 5 | -1 | 5 |
O3,O2 | 4 | 1 | -2 | 3 | -2 | 3 | 1 | 0 | -1 |
O3,O4 | -2 | 5 | -2 | 0 | -5 | 4 | 5 | 2 | -2 |
O3,O5 | -2 | -5 | -4 | 3 | -2 | -3 | 1 | 1 | -2 |
O4,O1 | -4 | 1 | 0 | -4 | 4 | -5 | -5 | 1 | -4 |
O4,O2 | 5 | 2 | 5 | -5 | 3 | -4 | -4 | -2 | -5 |
O4,O3 | -5 | 0 | 3 | 1 | 4 | -1 | 2 | 5 | 1 |
O4,O5 | -1 | -3 | 3 | -5 | 3 | 2 | -5 | -5 | -5 |
O5,O1 | 1 | 5 | 2 | 5 | 1 | 5 | 0 | 0 | 1 |
O5,O2 | -3 | -4 | 0 | 4 | 5 | 0 | 1 | -4 | 2 |
O5,O3 | -5 | 0 | 5 | 3 | -1 | 4 | 2 | 4 | 2 |
O5,O4 | 3 | 5 | -5 | 5 | 1 | -4 | -1 | 3 | 2 |
obj1?=element1
2n(m-1) relations
m : n
© 2017, Dr. Ling Ma
© 2017, Dr. Ling Ma
Machine learning:
Classifier: 3 * Feature1 ^ Feature2 + Feature3 / Feature4
Feature1 | Feature2 | Feature3 | Feature4 | Type | |
Obj 1 | 1 | 0 | 20.5 | a | Type A |
Obj 2 | -1 | 1 | 40 | b | Type B |
Feature1 | Feature2 | Feature3 | Feature4 | Type | |
Obj 1,001 | ... | ... | ... | ... | Type ? |
Obj 1,002 | ... | ... | ... | ... | Type ? |
© 2017, Dr. Ling Ma
Object | Type |
Obj 1 | Type A |
Obj 2 | Type B |
Obj 3 | Type C |
Deep learning:
Object | Type |
Obj 10,001 | Type ? |
Obj 10,002 | Type ? |
Obj 10,003 | Type ? |
© 2017, Dr. Ling Ma
© 2017, Dr. Ling Ma
Supplementing information:
Filling in missing objects or fixing their shape caused by occlusion. such as:
© 2017, Dr. Ling Ma
© 2017, Dr. Ling Ma
Dr. Ling Ma
Technion - Faculty of Civil and Environmental Engineering
SeeBridge WP4 Mid-term Report
Georgia Tech
Additional Data
© 2016, Dr. Ling Ma
Manual input & load from other source
Enhanced Geometry and Spatial Feature
© 2016, Dr. Ling Ma
Spatial Relationship Matrix
© 2016, Dr. Ling Ma
Transverse beam & Abutment
Uniqueness Checking
© 2016, Dr. Ling Ma
Element pairs are unique in terms of the relational features are unique
Other Attempts
© 2016, Dr. Ling Ma
Rule-based object identification
Similarity by feature matching
(Determinate result)
(Result with confidence)
Run-time file enrichment
Document-oriented database
Schema-free BIM model
Cloud service
(Deliver a model)
(Information sharing and collaboration)
The Root of Object Identification
© 2016, Dr. Ling Ma
Feature Matching
© 2016, Dr. Ling Ma
Longitudinal | Transverse | Vertical | Convex | ... | |
Primary Girder | 1 | -1 | -1 | -1 | |
Transverse Beam | -1 | 1 | -1 | 1 | |
... | 0 |
Primary Girder = (1,-1,-1,-1)
Longitudinal | Transverse | Vertical | Convex | |
Object 1 | -1 | 1 | -1 | 1 |
Object 2 | 1 | -1 | -1 | -1 |
Object 3 | 1 | -1 | -1 | 1 |
Object 1 = (-1,1,-1,1)
Primary Girder | Transverse Beam | |
Object 1 | 180 degree | 0 degree |
Object 2 | 0 degree | 180 degree |
Object 3 | 60 degree | 90 degree |
Pairwise Features
© 2016, Dr. Ling Ma
Closer to the transverse ?
Pairwise feature matrix
Pairwise Feature Matching
© 2016, Dr. Ling Ma
f1 | f2 | f3 | |
---|---|---|---|
E1,E1 | 1 | 1 | 1 |
E1,E2 | -1 | 0 | 0 |
E1,E3 | -1 | -1 | 0 |
E2,E1 | 0 | 0 | 0 |
E2,E2 | 1 | 1 | 0 |
E2,E3 | 0 | 0 | -1 |
E3,E1 | -1 | -1 | -1 |
E3,E2 | 0 | -1 | 0 |
E3,E3 | 0 | 0 | -1 |
f1 | f2 | f3 | |
---|---|---|---|
O1,O2 | 1 | 1 | 1 |
O1,O3 | -1 | 1 | -1 |
O1,O4 | -1 | -1 | 1 |
O1,O5 | 1 | 1 | 1 |
O2,O1 | 1 | 1 | 1 |
O2,O3 | -1 | -1 | 1 |
O2,O4 | 1 | 1 | 1 |
O2,O5 | 1 | 1 | 1 |
O3,O1 | 1 | 1 | 1 |
O3,O2 | -1 | -1 | 1 |
O3,O4 | -1 | -1 | 1 |
O3,O5 | 1 | 1 | -1 |
O4,O1 | 1 | 1 | -1 |
O4,O2 | 1 | 1 | -1 |
O4,O3 | -1 | -1 | -1 |
O4,O5 | -1 | -1 | 1 |
O5,O1 | -1 | 1 | -1 |
O5,O2 | -1 | 1 | -1 |
O5,O3 | -1 | 1 | -1 |
O5,O4 | 1 | -1 | 1 |
3 types of bridge elements (e.g., girder, sheer key etc.)
5 objects in the model
3 types of features
E1,E1 | E1,E2 | E1,E3 | E2,E1 | E2,E2 | E2,E3 | E3,E1 | E3,E2 | E3,E3 | |
---|---|---|---|---|---|---|---|---|---|
O1,O2 | 1 | 3 | -3 | 1 | -5 | 2 | -3 | 2 | 1 |
O1,O3 | -5 | 1 | -3 | 2 | -3 | -3 | 2 | 2 | -2 |
O1,O4 | -2 | 3 | -3 | 5 | 3 | -1 | -5 | -2 | -2 |
O1,O5 | -1 | 5 | 1 | 5 | 2 | 1 | 3 | 2 | 1 |
O2,O1 | -3 | 4 | 2 | 0 | 1 | -1 | -4 | 3 | -5 |
O2,O3 | -1 | 3 | 5 | -5 | 0 | 0 | -3 | 4 | -2 |
O2,O4 | 1 | 3 | -4 | 4 | 0 | 4 | -1 | 5 | -5 |
O2,O5 | -5 | -2 | -1 | 0 | 1 | 3 | 2 | 0 | 1 |
O3,O1 | -3 | 5 | 0 | -2 | -3 | 1 | 5 | -1 | 5 |
O3,O2 | 4 | 1 | -2 | 3 | -2 | 3 | 1 | 0 | -1 |
O3,O4 | -2 | 5 | -2 | 0 | -5 | 4 | 5 | 2 | -2 |
O3,O5 | -2 | -5 | -4 | 3 | -2 | -3 | 1 | 1 | -2 |
O4,O1 | -4 | 1 | 0 | -4 | 4 | -5 | -5 | 1 | -4 |
O4,O2 | 5 | 2 | 5 | -5 | 3 | -4 | -4 | -2 | -5 |
O4,O3 | -5 | 0 | 3 | 1 | 4 | -1 | 2 | 5 | 1 |
O4,O5 | -1 | -3 | 3 | -5 | 3 | 2 | -5 | -5 | -5 |
O5,O1 | 1 | 5 | 2 | 5 | 1 | 5 | 0 | 0 | 1 |
O5,O2 | -3 | -4 | 0 | 4 | 5 | 0 | 1 | -4 | 2 |
O5,O3 | -5 | 0 | 5 | 3 | -1 | 4 | 2 | 4 | 2 |
O5,O4 | 3 | 5 | -5 | 5 | 1 | -4 | -1 | 3 | 2 |
obj1?=element1
2n(m-1) relations
m : n
© 2016, Ling Ma
Similarity Instead of Determinate Result
O1 | E1 | E3 | E2 |
O2 | E2 | E1 | E2 |
O3 | E2 | E1 | E3 |
O4 | E2 | E1 | E3 |
O5 | E1 | E3 | E2 |
1st Candidate | 2nd Candidate | 3rd Candidate |
Conclusion
© 2016, Dr. Ling Ma
Machine learning for more complex cases
Tested on synthetic cases
SeeBridge WP4 Mid-term Report
Georgia Tech
Dr. Ling Ma
Technion - Faculty of Civil and Environmental Engineering
© 2016, Dr. Ling Ma
© 2016, Dr. Ling Ma
© 2016, Dr. Ling Ma
© 2016, Dr. Ling Ma
Requirement of the enrichment result
Produce feature-based enrichment rules
Algorithm for identification of model features and rule-based model processing
Technion + TUM + buildingSMART + AEC 3
Dr. Ling Ma
Technion - Faculty of Civil and Environmental Engineering
© 2016, Dr. Ling Ma
© 2016, Dr. Ling Ma
Element Type
Element Family
Element Property Set
Element Property
© 2016, Dr. Ling Ma
and many more...
?
© 2016, Dr. Ling Ma
© 2016, Dr. Ling Ma
Existence:
<TemplateRules operator="and">
<TemplateRule Parameters="PredefinedType=GIRDER;PropertySetName=Pset_SeeBridgeGirderPropertySet;Properties=IfcPropertySingleValue;SingleValuePropertyName=P_SeeBridgeGirderProperty1;SingleValue=IfcText;" />
<TemplateRule Parameters="PredefinedType=GIRDER;PropertySetName=Pset_SeeBridgeGirderPropertySet;Properties=IfcPropertyBoundedValue;BoundedValuePropertyName=P_SeeBridgeGirderProperty2;SetPointValue=IfcReal;" />
<TemplateRule Parameters="PredefinedType=GIRDER;PropertySetName=Pset_SeeBridgeGirderPropertySet;Properties=IfcPropertyEnumeratedValue;EnumerationValuePropertyName=P_SeeBridgeGirderProperty3;EnumerationValue=IfcText;" />
</TemplateRules>
© 2016, Dr. Ling Ma
© 2016, Dr. Ling Ma
Technion - Faculty of Civil and Environmental Engineering
© 2016, Ling Ma
© 2016, Ling Ma
and many more...
?
© 2016, Ling Ma
Element Type
Element Family
Element Property Set
Element Property
© 2016, Ling Ma
© 2016, Ling Ma
A physical element instance
A property set
An element type
© 2016, Ling Ma
Technion - Faculty of Civil and Environmental Engineering
© 2016, Ling Ma
© 2016, Ling Ma
© 2016, Ling Ma
Near
Overlap
Touch
© 2016, Ling Ma
© 2016, Ling Ma
Wider
Perpendicular
Parallel
Longer
Bigger
© 2016, Ling Ma
© 2016, Ling Ma
A | B | C | D | E | |
---|---|---|---|---|---|
A | 0 | 3 | 0 | 2 | 1 |
B | 2 | 0 | 0 | 3 | 1 |
C | 0 | 0 | 0 | 0 | 4 |
D | 1 | 4 | 0 | 0 | 0 |
E | 1 | 2 | 4 | 0 | 0 |
Matrix 1: Perpendicular
A | B | C | D | E | |
---|---|---|---|---|---|
A | 0 | 0 | 1 | 0 | 0 |
B | 0 | 0 | 1 | 0 | 0 |
C | 2 | 1 | 0 | 2 | 0 |
D | 0 | 0 | 1 | 0 | 2 |
E | 0 | 0 | 0 | 2 | 0 |
Matrix 2: Parallel
A | B | C | D | E | |
---|---|---|---|---|---|
A | 0 | 1 | 4 | 1 | 2 |
B | 0 | 0 | 0 | 3 | 1 |
C | 0 | 0 | 0 | 3 | 4 |
D | 0 | 0 | 0 | 0 | 2 |
E | 0 | 0 | 0 | 0 | 0 |
Matrix 3: Above
Rating score of being the determinant relationship for identifying A
© 2016, Ling Ma
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | |
---|---|---|---|---|---|---|---|---|
1 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 1 |
2 | 0 | 0 | 1 | 0 | 1 | 0 | 1 | |
3 | 0 | 0 | 1 | 1 | 0 | 1 | ||
4 | 0 | 0 | 1 | 0 | 0 | |||
5 | 0 | 1 | 1 | 0 | ||||
6 | 0 | 1 | 0 | |||||
7 | 0 | 0 | ||||||
8 | 0 |
Matrix 1: Perpendicular
Matrix 3: Above
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | |
---|---|---|---|---|---|---|---|---|
1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 |
2 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 1 |
3 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 1 |
4 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
5 | 1 | 0 | 0 | 1 | 0 | 1 | 1 | 0 |
6 | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 0 |
7 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
8 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
© 2016, Ling Ma
1 |
---|
2 |
3 |
4 |
5 |
6 |
7 |
8 |
A |
---|
B |
C |
D |
E |
Element A's Feature
© 2016, Ling Ma
SeeBIM 2.0 : a Cloud-based Platform for Information Collaboration on BIM model
SeeBridge Workshop @ Cambridge
Dr. Ling Ma
Technion - Faculty of Civil and Environmental Engineering
ionlny0irvononeinyhSnfrctuoeao1eTvcopuirU2tatHbgienongesngrhd5eilzgeg9nlayne1rsfdiec
Technology global 2015 engineering University...
Huazhong University of Science and Technology ranked top 19 global university for engineering in 2015
Huazhong University of Science and Technology ranked top 19 global university for engineering in 2015
(Rank by US news 2015 and Thomson Reuters)
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
SeeBIM
Michael Belsky, PhD, “A FRAMEWORK FOR SEMANTIC ENRICHMENT OF IFC MODELS”, Civil and Env. Eng., Technion. 2015
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
Ease the information retrieval
SeeBIM
Michael Belsky, PhD, “A FRAMEWORK FOR SEMANTIC ENRICHMENT OF IFC MODELS”, Civil and Env. Eng., Technion. 2015
Extend the schema
© 2015, Dr. Ling Ma
Horizontal/Vertical Element ? ->
bounding box ->
major axis ->
parallel/perpendicular to the ground
© 2015, Dr. Ling Ma
Rationalize Domain Knowledge
Ease the information retrieval
SeeBIM
Michael Belsky, PhD, “A FRAMEWORK FOR SEMANTIC ENRICHMENT OF IFC MODELS”, Civil and Env. Eng., Technion. 2015
Extend the operand and operator
Extend the schema
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
© 2015, Dr. Ling Ma
©2016, Dr. Ma, Ling
© 2015, Dr. Ling Ma
Dr. Ling Ma
Technion - Faculty of Civil and Environmental Engineering
© 2016, Dr. Ling Ma
One in nine of the bridges in the US are structurally deficient
FHWA needs $20.5 billion annually to eliminate the bridge deficient backlog by 2028, but $12.8 billion is being spent
Numbers from 2013 ASCE report card for America's infrastructure
607,380 National Bridges in the US !!!
Over two billion trips are taken daily across these bridges
© 2016, Dr. Ling Ma
14,700 bridges in Georgia
42 Inspectors in GDOT
Descriptive Inspection Report
AASHTOWare Bridge Management System (BMS)
Good? |
Satisfactory? |
Desirable? |
Bridge Risk Prediction
© 2016, Dr. Ling Ma
3D Semantically rich bridge model (Bridge Information Model)
+
=
McGuire, B., et al., (2016) Bridge Information Modeling for Inspection and Evaluation. Journal of Bridge Engineering
Average age: 42 years
Numbers from 2013 ASCE report card for America's infrastructure
Time consuming
Infravation - SeeBridge
© 2016, Dr. Ling Ma
© 2016, Dr. Ling Ma
Data Collection
Videogrammetry
(GoPro4 black)
Mobile laser scanning
(Trimble MX7)
Terrestrial laser scanning
(Leica c10)
© 2016, Dr. Ling Ma
Georgia (3)
Data Collection
Cambridge (10)
Haifa (1)
3D Reconstruction
© 2016, Dr. Ling Ma
Semantic Enrichment
© 2016, Dr. Ling Ma
Bridge Defects
© 2016, Dr. Ling Ma