Reduction of Material Usage in 3D Printable Structures Using Topology Optimization Accelerated with U-Net Convolutional Neural Network

Today’s 3D printers allow the creation of very advanced structures from various materials, starting from simple plastics up to metal alloys. Since the printing time and amount of material used to create structures are considered very important in terms of cost and energy consumption, it is better to optimize structures for that particular application taking into account all the conditions. In the current work, U-Net convolutional neural network-based topology optimization method (TO) that allows to reduce the material usage and eventually reduces the cost of 3D printing is introduced. The results showed that the accuracy of the method is highly reliable and can be used for designing various 3D printable structures and it applies to any type of materials since properties of materials can be included in TO.


Introduction
With the rise of 3D printing technologies, possibilities for optimization of structures -made of different materials, such as plastics, composites, metals and alloys -moved to the next level [1]. As shown in Fig. 1, 3D printers are highly efficient in reducing material cost, due to their layered deposition of material approach compared to the traditional approach [2]. Although, when the latter is coupled with a mathematical method -topology optimization (TO), the reduction of material usage is further decreased, while keeping the stiffness of 3D printed structures at a required level.
TO is a method for optimizing the distribution of material within a given space for a given set of loads and boundary conditions. The purpose of TO is to determine the optimal distribution of the ma-terial in the design area, thus reducing the weight of the structure and material cost (i.e. filament cost) during 3D printing. This method is based on repeated steps of design analysis and updating [3], which finally provides the best layout possible. Although, when the design of solid structures is complicated and has multiple initial and boundary conditions, the computational time of TO drastically increases and it takes to get optimized results up to several hours. Such computational difficulty is mainly due to the iterative nature of solvers designed for a large system of linear equations obtained from finite element analysis [4].
There are several methods to improve the computational time of TO. One way is to apply parallel computing based on multi-core central processing units (CPUs) [5][6][7], and graphics processing units (GPU) [8][9][10]. Thus, the computationally expensive portions of TO algorithm, especially the parts related to finite element analysis (FEA), are run in parallel among the cores of several CPU or GPU processors. Such an approach was well-known and applied intensively until artificial intelligence (AI) methods started to advance. Thus, in the latest trends, we can notice that AI technologies and open source AI platforms, machine learning (ML) methods in advanced manufacturing processes have expanded significantly [11,12]. This trend is also noticeable in the field of topology optimization, where, with the help of trained neural networks, structural topology and properties such as strength, modulus of elasticity, strain, and stress fields are predicted [13][14][15]. In the work [14], a convolutional neural network (CNN) was utilized to train the intermediate topology layouts obtained by SIMPbased TO to solve a solid mechanics problem. To forecast the optimum structures using AI method, the TO solver was stopped at an intermediate point after a few iterations and further fed into the CNN model. Comparing the trained CNN model to the conventional SIMP method, it was discovered that the trained CNN model could predict the final topology optimized structures up to 20 times faster with a few uncommon pixel-wise alterations.
Wang et al. [15] introduced a CNN with perceptible generalization ability for TO. Encoding and decoding components of the neural network that was proposed by the authors enable downand up-sampling operations. They compared their results with SIMP method and showed that they can achieve about 96% accuracy in predicting optimized topology with the increasing number of CNN training samples. Nie et al. [16] proposed a new data-driven TO model, named TopologyGAN, which uses physical fields computed in the original material domain as inputs to a conditional generative adversarial network (cGAN) generator. They showed that TopologyGAN is much more effective in reducing the mean square error and mean absolute error compared to cGAN. In the generator, they also introduced a hybrid network, particularly, U-SE (Squeeze-and-Excitation)-ResNet. The results showed good accuracy but some structures were unphysical, therefore their model needed some improvements at the neural network level.
In the current work, we develop a CNN using U-net and test them for predicting the topology of a cantilever beam. We compare the computational cost of the proposed method compared to the conventional SIMP method. We also consider the proposed TO approach, in terms of reducing the material cost for 3D printing applications. Thus, the further sections are developed in the following order: firstly, we briefly discuss TO and its role in 3D printable design samples. Furthermore, we introduce CNN method used in the current study and continue with the results and discussion section. Lastly, conclusive remarks are provided and future works are discussed.

General knowledge
Topology optimization is usually achieved through numerical calculations, where the design domain is discretized by finite elements. In other words, the finite element method (FEM) is the main numerical tool. FEM-based topological optimization is classified as Isotropic Solid/Empty (ISE), Anisotropic Solid/Empty (ASE), and Isotropic Solid/Empty/Porous (ISEP) topologies. Among them, the most practical class is ISE, where FEM elements are considered either filled with a selected isotropic material or void [17,18]. There are several works that compare different methods in terms of computational efficiency [3,[18][19][20][21][22]. The use of solid structures based on isotropic materials is widespread in all sectors of the manufacturing and construction industries. Similarly, 3D printing of structures using a single material is the most reliable and practical, because a stronger bond between printed layers is easier to achieve compared to a multi-material approach.
TO problem can be described as a search for a material distribution that minimizes the objective function F under a volume constraint G i . The distribution of the material is described by a variable density ρ(x), which can take on the value 0 (void) or 1 (solid material) at any point in the design plane Ω. The optimization problem can be written in mathematical form as [3]: where u corresponds to the state equation. For simplicity of further notation, it can be assumed that the objective function can be calculated as an integral over a local function f(u(ρ),ρ). Also, since in real conditions there are several restrictions, M additional restrictions are included in the general formulation. So far, several methods have been developed for solving TO problems: the density approach, the level-set approach, the phase field approach, and the discrete approaches [3]. Several papers compare the advantages and disadvantages of each of these methods in terms of their computational efficiency [5][6][7][8]. Among them, the density approach method, particularly SIMP method (solid penialized isotropic material) is commonly used by researchers/engineers and it is now commonly used in commer-where C -compliance; U and F -global displacement and force vectors; K -global stiffness matrix; u e and k e -elements displacement and stiffness vectors; x -design variable vector; x min -minimum relative densities; N -number of FEA elements, p -penalty factor, V(x) and V 0 -volume of material and design domain, and f -volume fraction of material. The optimization problem can be solved using various iterative methods such as Optimality Criteria (OC) method, Sequential Linear Programming (SLP) method, or Moving Asymptotes (MMA) method [24].
Within the last decade, TO method has been significantly improved [3], and its application to solve complex problems has expanded significantly [25]. Various multi-physics problems such as enhancing heat transfer processes [26,27], reducing liquid and solid interactions [28], optimizing the thermoelastic behaviour of structures [29], and improving the geometry of electro-thermal mechanical drives [30,31] are addressed using TO approaches, where given geometries are numerically discretized in 2D or 3D domains and optimized using TO for better performance. Although, the application of TO on such problems with multiple interacting physical properties in 2D or 3D domains leads to a noticeable increase in computational complexity, and hence an increase in the computation time of TO analysis. Therefore, in the current work, we address this use and try to improve it using machine learning methods.

Design example for 3D printing
As an example, in the current research work, we decided to study the TO of the Messerschmitt-Bölkow-Blohm (MBB) beam as shown in Fig. 2. In fact, this is one of the well-known structures that cial software such as SolidWorks, COMSOL and ANSYS. Thus, a common TO problem based on the SIMP approach, where the goal is to minimize the degree of conformity (i.e., maximize the rigidity of the design), can be written as [23]: are previously studied in most of the works that include machine learning studies [14,16,17]. In this example, continuous fiber angle optimization was considered together with TO so that 3D printed structure has a strength depending on the printing direction. The structure can be further tested using a mechanical machine as shown in Fig. 3 [32]. In the particular example, the volume fraction f was set to 0.4, meaning 60% of the material compared to the original MBB beam was saved. It should be noted that, in the current studies, the main purpose is to use ML methods for acceleration of TO, and reducing the computational time. Furthermore, we discuss the material cost reduction that can be achieved by TO approach.
Here, we consider the half of MBB beam due to its symmetricity and apply TO on it to develop an optimized material distribution for given boundary conditions. As can be seen from a) part of Fig. 4, there is a force (F) applied to the beam, while it is fixed at the right lower corner. The results obtained from TO using SIMP method show that the overall contour/shape of the possible structure is achieved after 15 iterations, while an additional 48 iterations are required to obtain the final finely tuned binary result. In other words, it is clear how the final structure would eventually get shaped after 15 iterations, but the method further has to go through more iterations (i.e. up to 60) to get the final reduced material distribution. According to our studies based on an open-source SIMP algorithm from [23], for a computer with Intel Core i7-4510 at 2.00 Ghz, the time for one iteration is around 0.88 sec. In total, to run 80 iterations to get the final TO, the overall computational time is about 70.4 sec. Once again, it should be mentioned that the mesh size in the current study is 120×40, which is 4800 FEM elements only. Usually, for complex structures, element size might go up to several thousand and millions. Therefore, the importance of accelerating the computational time is actual and needs proper studies.   To accelerate this process of TO to obtain material distribution for a given boundary and initial constraints in a shorter time, the authors propose to use a machine learning algorithm using neural networks. In such a way, it is possible to improve the overall duration from designing structures to 3D printing process.

Main idea
U-net is a special type of convolutional neural network, which was introduced by the Faculty of Computer Science at the University of Freiburg, and is designed to solve problems related to image segmentation. It consists of two main stages: convolution and development. At each step of the first stage (convolution), the model generates N reduced (compressed or convolved) filtered versions of the image, and at each step, the number of filters will be doubled. The subsequent stage (sweep) is the inverse version of the convolution. At each step of this stage, the images will be increased in size and filtered, with the initial number of filters corresponding to the final number of filters of the 1st stage, and the number of filters is halved at each step. The key action in this model is to store and transfer intermediate images from the convolution stage to the unwrapping stage, with the result of the first convolution step being used in the last unwrapping step, the second in the penultimate one, and so on. The final solution to this problem is an image of the same size as the original one.

Intermediate stages, layers
As mentioned above, U-net is a special type of CNN, which consists of the following layers: convolution, pooling, dropout and up-sampling. The convolutional layer is a set of maps (a set of matrices) with trainable ts (in different sources it is called differently: a scanning core or a synaptic core), which have a small receptive field but pass through the entire depth of the input volume. Within the framework of this experiment, at each convolution step, the number of filters with the ReLU (Rectified Linear Unit) activation function will increase by two, with an initial number of sixteen. Further, at each sweep step, the number of filters will be halved. The last set of sixteen filtered images will be filtered one last time with the sigmoid activation function. (3)

(4)
The dropout layer is a regularization technique used in artificial neural networks to prevent overfitting. Essentially, it zeros out random values in the input dataset and increases the efficiency of machine learning algorithms.

First stage: Convolution
The architecture of the first stage corresponds to the usual architecture of a convolutional neural network. It consists of several alternating steps that conditionally indicate the depth of the model. The input data of the first step are based on two images: N-th iteration of the SIMP and the difference �� � � �� � , with ����, otherwise 0 ���� -probability conversation matrix � � -initial matrix before dropout � � � -matrix after dropout A pooling layer is a selective sampling process that aims to reduce the resolution of an input matrix. Within the framework of this experiment, horizontal and vertical steps equal to two (N = 2, M = 2) was used to select the area, and the maximum value of each area was assigned to the resulting matrix.
Up-sampling layer is the process of increasing the width and length of the input matrix by N and M times, respectively. In experiments, this matrix was filled in according to the following formula, where N = 2, M = 2: between N-th and the previous image. Each step starts by applying two reconvolutions with 3×3 kernels with an increasing number of filters and a ReLU activation function, with the intermediate dropout layer of 10%. This is followed by the layer of reduction with a step of 2×2. In this experiment, the initial number of filters was taken as 16 and for each step, it was doubled. Also, the final result of each step is saved for transfer to the next stage.

Second stage: Deconvolution
The deconvolution stage (second stage) is the inverse of the first stage and differs in that as input data for each step, in addition to the received data, the data previously stored in the convolution stage is used. At each step, two 3×3 reconvolutions occur with a fitting number of filters and ReLU activation function, with the intermediate dropout layer of 10%. This is followed by a layer of magnification with a factor of two vertically and horizontally. After the 4th, final iteration, the result obtained passes through the last convolution layer with a single filter and a sigmoid activation function. After rounding the result, a binary image with the original resolution is obtained, where 0 corresponds to the void, and 1 to the material.

Training data
To implement the above model, images from iterations of the SIMP model along with the expected result are needed. Synthetic data created by Sosnovik and Oseledet [14], using an automatic 2D and 3D topology generator -an open-source SIMP Topy algorithm [33] -was used for training the CNN. In the end, 10000 imaginary samples were generated and used, and 100 SIMP iterations were run for each given boundary and initial conditions, along with the expected result. In Table 1, three different samples of input data, particularly, N-th iteration, the gradient (difference of N-th and (N-1)th iterations), and the final topology layout are shown. The grid size of the samples generated is 40×40 as can be noted in Table 1, and each topology sample is a tensor of 100×40×40, based on the grid size and the number of iterations, which is 100.

Evaluation method
This evaluation method checks the identity of each pixel in the image. Conventionally, it is de- The coefficient of Jaccard or the intersection-to-union ratio is used to calculate the level of overlap between the predicted image and the true image.

(7)
Due to the fact that there are only two classes of interest, the original formula can be modified to the following view: In the scope of this experiment, it will measure how much predicted image differs from the desired image.

Results and discussions
At the initial stage of the study, two experiments are carried out to determine the effect of depth on the CNN model. Here, depth means the number of layers in the neural network. To fully characterize In all of the above formulas, ω tp is the number of instances of class t predicted as class p, and n x this is the total number of instances of class x in the original image.
Mean absolute error (MAE) is also used to evaluate the accuracy of the model. MAE is a measure of the error difference between two observations that reflect the same phenomena.   the problem of TO under consideration, two different depths with different sizes are studied: in the first experiment, the depth is taken to be 3, while in the second one, a maximum depth is set to 4. The maximum depth is limited by the image resolution and it is one unit less than the minimum logarithm of the resolution to the base of two. For this case (40×40), this value is equal to four. Also, each experiment has four sections, where the effectiveness of the choice of initial data is also investigated. To run the experiment, along with a uniform distribution, the Poisson distribution is also used. It should be noted that the Poisson distribution is taken with three different coefficients P (λ = 5), P (λ = 10) and P (λ = 30), and experiments are carried out for each coefficient. Table 2 gives the main initial parameters of all models. As can be seen from Table 2 Table  2, and as explained above they are doubled at the convolution stage and halved at the sweep stage.

���
Thus, the initial parameters for all experiments, as well as which data are used, and how the model is built are presented in Table 2. As a result, 8 different experiment results from 8 different CNN are received. Each model is evaluated by comparing it with different SIMP iterations. Thus, in this experiment, SIMP iterations from 5th to 80th, with an interval of 5 iterations, are selected as shown in Tables 3 and 4. To obtain a satisfactory average accuracy of 94%, 5/10 iterations are sufficient. Summing up the intermediate results, it can be noted that each of these iterations is tested using the presented CNN model, particularly U-net model.
Thus, the results of the first 4 experiments are listed in Table 3, and the next four in Table 4 below. Based on these tables, it can be concluded that the accuracy of the presented U-net model is high and it can be used to speed up SIMP method. It can also be noted that the minimum accuracy of the model equal to 94% was observed when using the fifth iteration. Compared to the previous studies by [15], our results are significantly accurate even using 10.000 datasets. Wang et al. [15] achieved an accuracy of 94% when the data set increased to 20.000. Therefore, our current U-net model is simpler in terms of neural network complexity, which requires less time to obtain the topology, is more accurate, and can performance better accuracy with less training data set. Moreover, the maximum accuracy of 99% for material distribution is achieved using 80th iteration. At the same time, it should be emphasized that in this case, changing the depth of the model had a minimal effect on the results. This can be explained by the fact that the image has a small size, and increasing the depth from 3 to 4 does not make a big difference, but increases the computational complexity and runtime of the  The binary and MAE of the studies that are obtained using (7)-(9) are 0.97 and 0.0312 respectively. Moreover, the accuracy results of our studies using U-net based CNN are compared with previous results from the literature as shown in Table 5. The comparative analysis is based on the same number of training data samples, but different CNN depths that result in the accuracy. Thus, as can be seen from Table 5, our CNN has less number of layers, which directly correlates with faster computational time. In terms of accuracy, our result is better than Wang et al. [15] and comparable with the accuracy of Sosnovik and Oseledet [14].
Furthermore, Fig. 5 shows the expected results (bottom row) and calculation results (top row) of the presented model for three different cases for the material distribution using TO. It should be noted that the accuracy in all cases shown is greater than 98%. Furthermore, such accurate results obtained using neural networks are also 3D printable and allow for acceleration of the development process of 3D printable structures.
The TO is highly reliable and helps to reduce the material cost for 3D printing while keeping the mechanical or other properties of the structures at the same level as the original one. In the examples presented in Fig. 5, the volume fraction of the TO is set to 50% and it allows to reduce the material usage by 50%. Such an approach with the faster method of obtaining topologically optimized structures using CNN, can give many advantages to engineers who use 3D printing technologies for the design and construction of advanced structures.
As a future work, the current CNN model will be extended and improved to develop structures in 3D domain, that are highly required for the engineering of advanced structures. In future work, the main goal will be to keep the computational time as low as possible, while increasing the accuracy and robustness of CNN models, including the applicability of such approaches in 3D printing.  [15] 10.000 8~89

Conclusions
This work is devoted to the development of a method for accelerating the topology optimization (TO) of 3D printable structures. TO is a reliable mathematical method that allows for reducing material usage while developing advanced structures that are 3D printable. Although, the computational time of TO becomes costly with the increasing complexity of the boundary and initial conditions of the structural analysis. Here, a machine learning method based on a convolutional neural network (CNN) is proposed to accelerate the computational time.
Thus, as an example, MBB beam is considered in the work. The model based on the U-Net architecture is presented to speed up the computation of the TO under consideration, which in turn, reduces the design and development time of the structure. The results of the study are checked for identity using the binary identity by comparing it with the conventional TO method SIMP. Hence, the results showed that CNN model could give 95% of accuracy for 10th iteration of SIMP. Furthermore, when they are compared at 80th iterations, the accuracy of CNN showed much more accurate results and it is about 98%. In addition, it is noticed that changing the depth from level 3 to level 4 does not improve the accuracy of the experiment, although it increases the computational complexity and task execution time. Therefore, it is concluded that by keeping the depth as low as possible, still highly accurate results can be achieved, but within a shorter time and it is very important in reducing the computational time. As shown in the results, the proposed method is highly accurate and can be used for the generation of optimized material distribution to various structures with a given boundary and initial conditions. Such structures are 3D printable, and highly efficient and less material costly compared to structures developed using subtractive manufacturing. Using TO method developed here, the amount of material to develop structures can be significantly reduced while maintaining its strength.