Concrete 4.0 ‐ Sustainable concrete construction with digital quality control

Concrete is a mass commodity with more than approx. 250 Mio. batches of concrete being produced every year in Europe alone. Despite this enormous repetition rate, both the quality inspection of the concrete raw materials as well as of the mixed concrete are still primarily batch‐based, relying on manual processes, thus hindering the application of modern process control schemes. The reasons for this lack are on the one hand to be seen in the lack of suitable sensor technologies and on the other hand in missing control algorithms. The paper at hand gives an overview on the current state of the art sensor technologies for both raw material as well as concrete properties measurement. Further, it outlines a pathway towards an online concrete process control.


Introduction
With the Paris and Glasgow agreements, the international community has set binding action targets and implementation instruments for global climate protection [1].The construction industry, and in particular the building materials industry, plays a decisive role in achieving these climate and environmental protection targets.In order to reduce the worldwide CO2-emissions during the production of concrete, the use of composite cements containing three or more main constituents (including industrial wastes) and/or recycled aggregates is gaining significant importance, however leading to significantly more complex mixtures.The increasing complexity results in a strongly increased sensitivity to variations of the raw material properties and dosages as well as from the production-related side-conditions (deviations from the target composition, temperature and moisture conditions).As a result, concretes with recycled aggregate and composite cements are often less robust than conventional concrete, especially due to variations in water content and water demand of the aggregates [2].In contrast, increasing the robustness of such concrete mixtures can be achieved in different ways: a.
Reducing the variations in raw material properties, e.g. by sorting recycled aggregates into different fractions.This approach however yields in higher costs, due to the sorting process.Further, aggregates removed due to insufficient quality normally have to be deposited as waste, thus not being ecologically favourable nor economically feasible.b.
Compensating the negative effects of fluctuations in material properties by increased safety margins, such as a significant increase in cement, binder or powder content [3][4][5].However, this is neither economically nor ecologically justifiable [6,7].c.
Using new types of quality control methods, which are able to react to and compensate for fluctuations in the raw materials (such as the recycled aggregates), in the mix or in the processing conditions.
The focus of the paper at hand is on the third approach (c).To this end, we propose to continuously monitor the raw material properties (input) and to quantify the concrete properties (output) in real-time using newly developed sensor systems.In this way, for example, the influence of variations in recycled aggregates -but also other concrete raw materials -on the end product (the fresh concrete) can be quantified and in a next step compensated for without negatively affecting the economic efficiency and environmental balance.In this context, a concept and associated methods for digitization of the concrete process chain using computer vision and artificial intelligence are presented in this paper.
We begin by giving a review on the current state of the

Abstract
Concrete is a mass commodity with more than approx.250 Mio.batches of concrete being produced every year in Europe alone.Despite this enormous repetition rate, both the quality inspection of the concrete raw materials as well as of the mixed concrete are still primarily batch-based, relying on manual processes, thus hindering the application of modern process control schemes.The reasons for this lack are on the one hand to be seen in the lack of suitable sensor technologies and on the other hand in missing control algorithms.The paper at hand gives an overview on the current state of the art sensor technologies for both raw material as well as concrete properties measurement.Further, it outlines a pathway towards an online concrete process control.
digitization of the concrete production chain in Sec.

Related work and Background
In many manufacturing industries, automation and digitization have enabled a strong increase in productivity in recent decades.However, productivity in the construction industry stagnated during this period [8].The construction industry -and here especially the concrete sector -is still one of the least digitized industries of the global economy.This is often argued with the high individuality of buildings and structures and thus the lack of serial production.Considering the building materials sector -and here especially concrete -this however is not true.In Europe alone (incl.Turkey; data 2018) more than 380 Mio.m³ of ready-mix concrete are produced [9], corresponding to nearly 700.000 mixes per day (mean batch size 1.5 m³).Further, in Germany alone (54 Mio.m³ annually) more than 80 % of ready mixed concrete belong to strength classes ≤ C30/37, thus having very similar mix compositions [9].This results in extremely high repetition rates, which offer great potential for the use of digital methods (in particular of methods based on machine learning) to identify quality outliers and to learn from quality deviations, e.g. by an inline adaption of the mix composition.Such methods, which use sensor data and models as to control industrial processes are referred to as Industry 4.0 approaches.
The process chain of producing concrete structures includes the planning phase, the production phase, the manufacturing phase and the maintenance.Digital processes are already integrated in all phases, but the degree of digitization varies greatly.Methods for digital design and planning of structures (CAD, BIM etc.) are an integral part of the planning phase [10,11] and BIM-based augmented reality systems are slowly gaining ground [12,13].Further, in recent years, various approaches for the additive manufacturing of concrete structures have been developed and some large-scale demonstration projects have been successfully completed [14][15][16].The key advantage of additive manufacturing or 3D printing of concrete is the high degree of automation and the possibilities for individualization.In addition, 3D printing technologies only uses material where it is structurally or functionally necessary, thus being extremely efficient the use of resources [17].
With the technologies described above, the planning phase and the production phase can be directly linked digitally.However, a clear lack in the application of digital methods can be seen in the concrete production sector.As the characterization of the properties of raw materials and the quality control of the concrete production process are still based on conventional, batch-based, non-digital methods, currently the high repetition factors outlined before cannot be used and translated into improved, environmentally optimized robust mixes.For this reason, an increasing interest has emerged in developing and providing methods for digitization in the concrete production industry.Regarding the prediction of concrete properties from its mix design, artificial neural networks have been successfully employed [18,19], but mainly focus on properties like compressive strength and are only based on the nominal composition design of the concrete so far, thus neglecting the actual properties of the raw materials and the actual composition.The use of sensorial information on the raw material properties is not considered yet for the prediction and control of (fresh) concrete properties.Looking at the monitoring of raw material, in [20] a method for the classification of individual recycled aggregates based on convolutional neural networks (CNN) was proposed.However, the data requires the particles to be separated from each other, which is an unrealistic setting in practice and completely disregards properties like the particle size distribution, which has an essential impact on the concrete properties.
Looking to the properties of the final product of the mixing process, i.e. fresh concrete, also here testing is entirely batch-based using empiric testing methods.A research approach for fresh concrete quality monitoring was proposed in [21], where the concrete mix proportion is determined from images of fresh concrete using a convolutional neural network (CNN).In [22], an approach for determining the workability from image sequences acquired during the mixing process using a LSTM deep learning network has been postulated.While promising results were obtained, processing was done on rather low resolution grey scale images only and the approach relied on 2D transformations, ignoring the clearly visible effects perspective distortions.

Digitized concrete production chain
In the transition process towards a more sustainable highquality concrete construction industry, essential process steps in the concrete process chain must be digitally mapped.In the opinion of the authors, this comprises (i) a continuous monitoring using automated sensor technologies of the relevant input parameters influencing the concrete properties such as raw material properties, environmental and production influences, as well as (ii) a sensor monitoring of the result of the involved processes (output), such as the fresh concrete properties after mixing or delivery at the construction site.Monitoring both input (e.g.raw material properties) and output (e.g.fresh concrete properties) will enable to establish a control-loop.For establishing this loop (iii) deep learning based control schemes are needed, with which the concrete properties can be predicted based on the concrete raw material properties and the mix composition and appropriate countermeasures can be defined in real-time in case of deviations between the prediction and the measured properties.
With this 3-fold approach, the authors believe that the usage of recycled raw materials, composite cements and environmentally optimized concretes can be greatly increased without increasing the risks going along with that.At the same time, using such technologies will allow for a reduction of the large safety margins in the mix development of concrete, which are typically applied in order to counteract fluctuations in the raw material properties and in the processing conditions.
The proposed concept significantly extends the current state of the art in the concrete mixture development and concrete production process, which has so far been purely empirical.A key role in achieving this goal is played by a combination of contact and non-contact sensor systems, providing the data necessary for such a data-driven control loop.
An overview of the different AI based online monitoring methods addressed in this paper and their association to the individual process steps within the concrete production chain are shown Fig. 1.In this context, we differentiate between systems to be incorporated into the production process, i.e. in the concrete plant (Sec.3.1), and systems to be applied after production, i.e. at the construction site (Sec.3.2).While the first kind enables an online reaction and control of the concrete properties in real-time, namely already during the batch-based mixing of the concrete (inbatch adaptation), the latter approaches allow for a postproduction quality control (next-batch adaptation).All methods can be combined into a digital control loop for ready-mixed concrete (Fig. 1).
As shown in Fig. 1, the proposed sensor-based monitoring methods related to the concrete production process include the characterisation of the raw materials -here with a focus on the aggregates (e.g. grain shape, grading curve, composition of recycled aggregate fractions) -and the continuous quantification of the fresh concrete properties (rheological parameters, mix homogeneity) during the mixing process.Corresponding control algorithms make it possible to adjust the concrete composition before mixing begins as to react to fluctuations in the raw material properties or to add suitable additives during the mixing process in case deviations from the desired properties are detected.The characteristic values determined by means of optical non-contact measurement methods are thus fed directly back into the process and allow direct intervention in the production process.The proposed methods for construction site concrete quality control cover the concrete transport, the discharge, and the quality inspection at the construction site.The focus here is particularly on fresh concrete properties (in-situ composition, consistency, segregation tendency, pumping properties).The gained sensor information can be used as decision support for the further concrete processing on the construction site or can be fed back into the production process for a self-learning mix adaptation (post-production control).

Digital quality control in the concrete plant
This section gives a brief overview on the proposed monitoring approaches that are developed by the authors to be applied during the concrete production process in order to enable an online control of the concrete properties.

Sensor based characterisation of concrete aggregates
Aggregates, i.e. fine and coarse particles usually of sizes between 0.1 and 32 mm, make up around 70-80% of the concretes volume.Due to the large share of aggregates, this type of raw materials significantly influences many important properties -both in the fresh and in the hardened state of the concrete.These include fresh concrete properties such as consistency, workability and segregation tendency as well as hardened concrete properties such as compressive strength, durability, etc.In this context, particularly the size distribution of the aggregates (formally known as grading curve) has a substantial effect on the properties and quality characteristics of the concrete [23].
As a consequence, in order to achieve the desired properties (e.g. a target consistency), the aggregate size distribution has to be closely considered during mix design and controlled during concrete production since it significantly affects the water demand or the amount of required superplasticizer.In practice however, the size distribution is usually determined on a regular, low-frequency basis (2-3 times per year) on small sample batches of the aggregates (a few kilograms) by manual mechanical sieving.Both variations in the batch the sieved sample was taken from as well as temporal variations during continuous production are thus neglected, despite being of extremely high relevance.. Consequently, the size distribution and composition of the actual aggregate used for individual production batches of concrete can vary greatly, however with the variations remaining undetected and with the origins of variations in the (fresh) concrete properties remaining unknown.This especially true for recycled aggregates, where not only the particle size distribution but even more the chemical/mineralogical composition of the aggregates highly fluctuates.
In order to overcome these limitations, the authors propose an approach for the image based prediction of the size distribution of concrete aggregates, delivering a realtime capable analysis of the size distribution of concrete aggregate.In this context, we make use of modern techniques based on artificial intelligence and use machine learning methods in order to learn the mapping between the images and the corresponding particle size distribution using convolutional neural networks (CNN) [24] and Vision Transformers (ViT) [25].Incorporating such an approach as online measurement process into the production chain of concrete by installing cameras above the aggregate feeding belt allows to derive knowledge about the possible property and quality fluctuations in the total amount of aggregates used for the particular concrete batch (Fig. 2).A detailed overview on the developed methodology as well as on the precision of image based methods compared to standard sieve testing is given in [24,25].Here it could be shown, that using image based methods, the grading curve of aggregates on a conveyor belt as shown in Fig. 2 (left), could be determined with an overall accuracy of less than 2%, thus outperforming classical techniques both in accuracy as well as (especially) speed.As such techniques allow for an inline-adaption (e.g. over the conveyor belt), they enable to react on detected variations e.g. in the size distribution of the aggregates in real-time by adapting the composition, i.e. the mix design of the concrete accordingly, so that the desired concrete properties are reached.
Figure 2 The proposed concept of the digitised concrete production chain enabling an automated and digital control loop for the ready mix concrete production towards a more sustainable concrete industry 4.0.

Image based monitoring of the mixing process
In current practice, the quality inspection of fresh concrete is mainly conducted offline, i.e. after the mixing and production process, using empirical test methods based on small batch samples taken from the concrete.However, at this stage of the production process, only very limited control of the concrete properties remains possible.For this reason, an online quality assessment during the mixing process is desirable, since it would enable real-time control of the concrete properties and an online reaction (during the mixing process) on potential deviations from the target properties.However, currently, the online quality assessment during the concrete mixing process is restricted to rough consistency estimations based on the electrical energy consumption of the mixer [26].In the opinion of the authors, this method in itself is not sufficient for a precise derivation of the complex rheological properties of fresh concrete as it only allows to determine one parameter, i.e. the dynamic viscosity η at one given shear rate (corresponding to the speed of rotation of the mixer).However, the fresh concrete properties are characterized by a great number of parameters, such as the Bingham yield stress and plastic viscosity (e.g.[27]), the thixotropy (e.g.[28]), the sedimentation and bleeding behaviour or setting behaviour.In the literature therefore mechanical probes to be installed in the mixer have been proposed, which however, are technically complex and nearly always result in prolonged mixing durations [29].A promising approach was presented Garrecht et al. [30] who vary the mixing speed thus obtained detailed insights in the concretes rheology.
In contrast, the authors propose to augment the electrical power measurements during mixing (which is a standard technology in concrete plants worldwide) with a video-optical monitoring of the mixing process as basis for a computer vision-based online derivation of the rheological properties of the fresh concrete.Starting from the hypothesis, that concretes with different rheological properties lead to different flow patterns during the mixing process, we investigate methods in order to solve the inverse problem, namely to infer the concretes fresh properties from camera observations of the mixing process.More specifically, we make use of a stereoscopic camera setup, allowing the reconstruction of the 3D concrete surface, carrying valuable additional information related to the fresh concrete properties.Based on the gathered video-data a CNN is being developed and trained in order to infer characteristics from the three-dimensional image data of the flowing concrete in the mixer.In its current implementation, the 3D surface of the fresh concrete is determined as a function of time from the acquired stereo-image sequences using classical image mapping methods [31].Then, both, the acquired image sequences and the determined 3D information are used as input data for the CNN, which learns the extraction of a feature embedding of the recorded image and depth frames.The regression performed in the CNN finally produces values for viscosity and yield stress of the fresh concrete.For methodological and mathematical details on the described approach, we refer the reader to [32].
Building upon the proposed method for online concrete production monitoring, strategies can be applied that specifically control and adjust the concrete towards its target rheological properties, e.g. by developing a suitable concept of chemical additives that are added to the mixing process.The technical opportunity to determine the rheology of fresh concrete in real-time makes it possible to iteratively develop self-learning algorithms for controlling the concrete properties and, thus, to enable digital control and monitoring of the whole production process.

Digitized quality control on construction site
In order to gain precise control over the concrete properties, it does not suffice to digitally map the concrete production process only.In addition to the previously described methodology to be implemented in the ready mix plant, digital quality control methods on the construction site are necessary.The digital test methods -here defined as automated contact-less digital determination of the concretes properties -must provide a much deeper insight into the concretes properties, i.e. must ideally detect all relevant fresh concrete properties.To this end, the authors developed various image-based quality control methods, which can be seamlessly integrated into the quality acceptance test scheme.

Image-based methods for evaluating the rheology properties at concrete discharge
In this section, we present a method for determining fresh concrete properties right during the concrete's delivery.Similar attempts have been made by correlating the energy consumption for rotating the mixing drum of a truck mixer to values obtained by rheometer tests or using a concrete mixing truck itself as a rheometer [33,34].Nevertheless, these methods have in common, that they require substantial technical modifications on the mixing truck, thus limiting these techniques to the truck owner.Besides such truck mixer based systems, rheometer test methods have gained attraction in testing of fresh concrete [35].However, these test methods today are exclusively batch-based, laborious and the data interpretation is highly challenging [36].
In order to enable a concrete characterisation at the time of delivery and to improve concrete quality control and safety assurance on construction sites, we propose a novel test procedure for the automatic on-site characterisation of fresh concrete during the discharge process of a mixing vehicle (Fig. 3).To this end, we developed a computer vision based strategy for a digital characterisation of fresh concrete based on image sequences showing the concrete flow at the mixing truck's discharge.Given the recorded video frames, we make use of a deep learning approach based on Vision Transformers (ViT), for the prediction of concrete properties like consistency and rheological parameters.The approach is built on the premise that fresh concretes with different rheological properties exhibit a different and distinguishable flow behaviour; a hypothesis which is founded on fluid mechanical formulations of non-Newtonian fluid flow describing the flow behaviour of Bingham fluid's (such as concrete) as a function of its rheological parameters, namely the plastic viscosity and yield stress .We tackle the inverse problem, namely the objective of predicting the rheological properties and the consistency of fresh concrete from observations of its flow behaviour and make use of image sequences showing the concrete flow during the discharge process of a mixing vehicle as input data to our approach.Finally, a Vision Transformer based method, which is able to model temporal relations within sequential data, is learned as mapping function between the image data and the rheological properties, such as yield stress and plastic viscosity.For more details on the methodology, we refer the reader to [37].
Figure 3 High level overview on our approach for fresh concrete characterisation.Image sequences are recorded at the concrete discharge channel of a mixing truck.The proposed Concrete Flow Transformer is used to predict the consistency and the rheological properties of the fresh concrete.

Image-based methods for evaluating concrete properties
As outlined before, the fresh concrete properties are much more complex than considering the consistency or the rheological properties only.Whereas e.g. in [38] a method has been presented, to extract rheological parameters by observing the spread-flow behaviour of the concrete, methods as to extract e.g.composition data are missing or are highly labour intensive.
At the construction site during acceptance testing, the construction companies rely on delivery-slip data combined with slump testing only, in order to verify, that the concrete complies to the desired properties.Looking to classical slump testing, an experienced technologist can gather an abundance of additional information by visually inspecting the slump cake, e.g. the maximum grain size of the mix, the homogeneity with which aggregates are distributed in the slump cake, water separations, and segregations, however without being able to quantify any of these effects.
In contrast image-based methods allow to extract and quantify these parameters.In our work, we implement photogrammetric computer vision and CNN based algorithms, to correlate optical patterns in the visual data with concrete technological properties determined using standard empirical tests.In a first step, a 3D-surface topography of the spread out fresh concrete can be calculated using Multi-View Stereo (MVS) reconstruction (Figure 4).Furthermore, a semantic segmentation of the classes table, suspension and aggregate is performed using the approach of [39,40].In this way, a large number of concrete properties (e.g. the paste content, the grain size distribution (> 4 mm) or the maximum grain size) can be digitally evaluated as part of the slump flow or flow table test on the construction site [41].The digital data of the concrete properties obtained in this way can be directly integrated into a digital quality control loop.The concrete manufacturing plant is directly supplied with data of concrete properties from the construction site.Quality deviations detected at the construction site can be compensated for efficiently in terms of time during further production.

Summary
This paper gives an overview on the potentials of digital methods in concrete production and quality control, spanning from concrete raw materials up to the final concrete product.The methods presented in this paper are based on photogrammetric computer vision and deep-learning algorithms.During concrete production, the information obtained by these methods (e.g.grading curve, rheological properties of fresh concrete) can be used to make specific modifications to the concrete composition or to optimize the concrete properties by applying an additive concept during the mixing process.Concrete discharge and the quality control on the construction site can additionally be assessed by means of image-based methods.
In this way, information can be provided for a digital control loop and a self-learning concrete composition development.
In the opinion of the authors, these technical solutions will significantly reduce the possibility of human errors and will make it possible to ensure sustainable and high-quality concrete construction in the future.In addition, materials can be integrated into the concrete production process that were previously classified as unsuitable due to excessive material fluctuations (e.g.recycled aggregate).The methods presented in this paper provide tools that address this complex problem and provide a solution approach, so that complex eco-friendly concrete mixtures can be produced accurately, using recycled materials and meeting the highest quality standards.

Figure 1
Figure 1The proposed concept of the digitised concrete production chain enabling an automated and digital control loop for the ready mix concrete production towards a more sustainable concrete industry 4.0.

Figure 4
Figure43D models of spread fresh concrete with different flowability (different flowability results from different paste contents)[41] 2, followed by an overview on our conceptual design towards the digitization of the concrete production chain in Sec. 3. Building up on that, the developed methods for a real-time monitoring of the production process are described, including the sensor-based characterisation of raw aggregate material (Sec.3.1.1)and the AI based determination of rheological properties already during the mixing process (Sec.3.1.2).Finally, we provide an overview on image-