A 3D-Printed Computer

. This


Introduction
The von Neumann architecture, where a computer is constructed from separate units for memory, arithmetic, and program control to perform computations sequentially, has been the engine behind the digital transformation. [1] Within this architecture, computation capability is increased through running the circuits at higher speeds or by adding computation cores and on-chip memory. The approaching physical limits in semiconductor manufacturing, the inherent requirement for sequential processing, and suboptimal performance while implementing certain algorithms, among other deficiencies, have motivated the research for unconventional computing methods such as neuromorphic or quantum computing. [2,3] Two rapidly growing areas that rely on unconventional computation methods are in-sensor and near-sensor computing. [4] Various forms of wireless sensor networks, such as wearables and Internet-of-Things (IoT) modules, use sensors to monitor their surroundings. The data produced by these sensors are often used to create contextual information such as user activity or machine health monitoring. In contrast to brain simulators, cognition in sensing applications is about distinguishing between a limited number of situations of interest. Complex machine learning algorithms used in these applications need to process the mostly redundant data, either locally or remotely, adding a significant power burden during the processing and communicating the sensor data. The recent efforts to develop neuromorphic or in/near-sensor computing methods aim to utilize the physics and structure of the materials and devices to limit the reliance on general-purpose processors. Besides the savings on energy usage, this approach enables a real-time, context-based response from the sensor module when needed without reliance on communication networks or remote computers.
Reservoir computing (RC) is a computational framework that is well suited to physical implementations of contextual computers. [5,6] The computational core of a reservoir computer is made of nonlinear neurons with both feedback and feedforward connections, resembling a recurrent neural network (RNN). The connections between the (physical) neurons occur through some sort of information (i.e., energy) exchange between them. The feedback connections in RNN and RC provide a notion of time and memory, making them suitable for processing time-series data. Contrasting an RNN, the weights of the couplings between the neurons in an RC are fixed, and the training is conducted only on the weights of the output layer. This makes the training of RC much simpler and more robust than RNNs, typically requiring simple algorithms such as linear regression. Unlike the traditional von Neumann architectures, the merging of memory with computation units means reservoir computers do not need separate memory to store or communicate data. [4] In physical realizations of RCs, the material or device physics are employed to nonlinearly map input data onto a high-dimensional space where different events may be distinguished from each other. Physical RC systems have been developed based on mechanical oscillators, [7][8][9] memristors, [10][11][12] photonic DOI: 10.1002/aisy.202300015 Vast amounts of data are generated by sensors that are used to monitor people, animals, plants, machines, structures, and the environment. Increasingly, this data is used to create relevant context based on sophisticated pattern recognition algorithms trained using past labeled data. However, most of these sensor systems are severely constrained regarding their communication and computation capabilities due to limitations on available energy, size, or location. New computational approaches are needed to overcome the limitations of existing digital processors in contextual processing. This article discusses the development of the first such computer that is entirely made based on common 3D-printing materials and techniques. It is demonstrated that a simple structure printed with regular 3D printers can be driven and used with common measurement tools to perform sophisticated contextual computations, including standard benchmarks and a demonstration of user activity detection from sensor data. The correlation between memory capacity, nonlinearity, and sampling rates with this computer is examined. The 3D-printed structure may be used as a stand-alone computer to detect patterns in general data streams. Moreover, the computer can be integrated with the sensorized 3D-printed structures, leading to the development of cognizant 3D-printed systems comprising sensors and contextual processors.
3D printing enables rapid fabrication of complex, multilayer, and even multimaterial structures at once using various printable materials. [18] Advances in 3D printing technologies and additive manufacturing have been utilized to develop 3D-printed sensors and hence, 3D-printed smart systems (i.e., structures with embedded sensing capabilities). [19] However, typical 3D-printed systems, including 3D-printed sensors, are generally passive components, void of computational capability. The desire to add computation capability in analog or digital domains has motivated the research on developing 3D-printed transistors and active components. [20] Much of this research has thus far focused on developing transistors for analog signal processing or conventional von Neuman digital processors. However, these transistors are printed at much lower densities than the existing silicon microelectronic chips, severely limiting their utility as computational elements. Alternatively, 3D-printed neuromorphic devices and processors have been proposed as potential methods to add computing power to 3D-printed structures. [21][22][23] 3D-printed optical signal processors have been proposed to circumvent the challenges on the electrical side for processing large quantities of input data. [24] Nonetheless, the existing solutions for signal processing with 3D-printed structures either face significant challenges in scaling up the computation capabilities or require sophisticated tools to recover the processed data.
In this article, we describe the design and operating principles of a contextual computer that is made through standard 3D printing with simple electrical ports for the input and output stages. Despite its simplicity, the computer can tackle complex standard tests and is used to solve the practical problem of user activity detection. Adding to the computational capability of the demonstrated computer is simply achievable by printing additional computational nodes. At a material cost of less than $1, this computer can be used next to existing smart systems for contextual signal processing. It can also be embedded within the structure of 3D-printed intelligent systems, enabling the realization of cognizant 3D-printed systems.

Results
Echo State Networks (ESN) form a particular group of RCs with requirements that can be met using physical components. Within an ESN, the reservoir should be built from coupled nonlinear nodes with fading memory. [5,25] The dynamic response of physical systems often satisfies the fading memory requirement for ESNs when the data are applied or collected at a sampling rate proportional to the response time of the system. The coupling requirement can be met by letting energy transfer from one device to the others nearby. Therefore, it is possible to build physical RCs that utilize various forms of nonlinearity and coupling.
The nonlinearity in the current-voltage response of regular thermistors (i.e., temperature-sensitive resistors) due to selfheating is a well-known phenomenon. Within the scope of RC, thermistors can be viewed as a particular form of memristors, where the thermal response of the material affects its electrical behavior. Researchers recently demonstrated an RC that was built using off-the-shelf thermistors that provided the nonlinear response and fading memory requirements while the coupling between the elements was achieved by directional sharing of the electrical current. [26,27] 3D-printed resistors exhibit interesting time-dependent, nonlinear responses. These resistors can be made, for instance, from a conductive material using fused deposition modeling (FDM) technology. FDM is presently the most common 3D printing technology, where a filament of the printed material is melted and deposited selectively using a printer head. The nozzle temperature for printing these conductive filaments is between 120 and 250°C, depending on type. [28] However, printed resistors exhibit significant nonlinear responses due to self-heating at much lower temperatures (50-60°C). Figure 1 shows the response of a 3D-printed resistor under different conditions. As can be seen, these resistors exhibit a nonlinear response under high currents due to self-heating. Additionally, the resistors exhibit a time-dependent response if the time spent at each current level is shorter than the time needed for reaching thermal equilibrium. The nonlinearity and time dependence in resistors' responses make 3D-printed resistors suitable candidates for developing RC computers based on ESN topology. The additional requirement for coupling between the nodes can be achieved in electrical or thermal domains. To achieve this, we printed several resistors close to each other such that the heat generated by one would reach and affect its nearby devices. The reservoir structure can be interpreted as a three-layer neural network, in which the weighted connections between the neurons are realized through thermal coupling. Moreover, the nodes on each layer are electrically coupled to each other. Three layers of conductive traces printed of carbon-polylactic acid (PLA) composite were stacked on top of each other with a gap in between, filled with pure PLA as the insulating material, as shown in Figure 2. The nonlinear, time-dependent responses of these resistors and the coupling between them satisfy the requirements for building an ESN. Large contact pads were used to apply the input signals or to read the data from specific nodes. The input layer is driven by an analogue shift register which shifts the input signal multiple times and maps the generated signals to the reservoir. The output layer is trained by running a linear regression on all the outputs from the reservoir.
Three samples with similar lateral dimensions that differed only in the thickness of the conductive layers were 3D printed. The thicknesses of the conductive traces in samples 1-3 were 2, 1, and 0.5 mm, respectively. These structures were initially studied to evaluate the nonlinearities and time dependences of the responses of neurons at different locations and the couplings between them. Figure 3 shows test results from these samples on individual neurons' responses to electrical excitations. It was observed that the responses of the neurons might be affected by injecting a control current, I C , into different nodes in the structure (Figure 3c). This effect may be used to bias different neurons postfabrication to make them respond differently from other similarly fabricated devices, adding another parameter that can be used to enhance the reservoir complexity. Figure 3e demonstrates the measured electrical and thermal responses of a resistor on the top layer of the 3D-printed processor over time due to a step current input. After the injection of the current, both the voltage across the resistor and its temperature slowly rise until thermal equilibrium with the environment is reached. In this case, the resistor exhibited a thermal time constant (TTC) of %62 s. The TTC will be different for different neurons in the structure and will depend on the thermal boundary conditions, such as being embedded within the structure or being exposed to the environment on one or more surfaces. This property, too, adds a degree of randomness and helps with using the 3D-printed structure as a contextual processor.
Reservoir richness (i.e., the combination of the number of neurons, their interconnections, and their memory capacity [MC]) determines its ability to tackle complex computations. The neurons in the printed reservoir exhibited both linear and nonlinear responses to different ranges of input signals, as expected. Figure 4 demonstrates some characterization results for sample 3. A single printed computer provides numerous possibilities to arrange input and output layers by applying or reading signals to different contacts (Figure 4a-c). Figure 4d illustrates thermal images of the reservoir at different instances of time after the application of an input to the reservoir, showing the evolution of reservoir characteristics through heating and its subsequent effect on the I-V characteristics of the neurons. Figure 4e shows samples of an incoming time-series signal, its time-shifted copies applied to the reservoir, and finally, the reservoir outputs. During normal operation, the RC distinguishes between different events after combining the reservoir outputs using weights calculated during the training stage. Video S1, Supporting Information, shows a thermal video of the computer during its operation in real time. Video S2, Supporting Information, includes the real-time measurements of the signals from the computer as it performs computations.

Performance Evaluation Based on Standard Benchmarks
The computational capability of the proposed processor to solve a nonlinear problem which requires a memory of past events can Figure 1. The current-voltage characteristic response of a free-hold 3D-printed resistor under different test conditions. In each case, the current through the resistor was increased in steps from zero up to the maximum value I max (indicated on the left) and back to zero. The current through the resistors was held constant for a period of Δt (indicated at the top) before measuring the voltage across the resistor and proceeding to the next step. It is noteworthy that under these test conditions, the resistor response is repeatable, indicating a permanent change to material response has not yet occurred.
www.advancedsciencenews.com www.advintellsyst.com be evaluated using a nonlinear autoregressive moving average (NARMA) task. [29,30] The goal of a NARMA task of order n (i.e., NARMAn) is to predict the next point in a time-series signal that is produced using a nonlinear combination of the past events, where n indicates both the order of the nonlinearity and the amount of memory required by the problem (see Experimental Section). The generated dataset includes 360 samples. Two hundred fifty samples from this data (%70%) were randomly selected and used to train the output layer weights. The remainder of the dataset was used to validate the computer performance. Figure 5a shows the response and the analysis of the response of the 3D-printed processor to NARMA tasks of varying order (n ¼ 1 : : : 10). This simple processor with only 18 nonlinear neurons demonstrates good performance for n ≤ 7. The performance of the processor can be improved through several simple approaches, including adding additional computational nodes, adding complexity through varying the control signal, or time-multiplexing the tasks between parallel computers. The NARMA test also helps study the compromise between the input signal's sampling rate and retainable memory with the system. [31] The system state approaches thermal equilibrium at a slow sampling rate (i.e., low data rate). Although the reservoir exhibits the most nonlinear response in this case, and hence the ability to solve complex problems, it may have lost information about distant past events. On the other hand, a fast sampling rate (i.e., high data rate) reduces the reservoir nonlinearity needed for contextual computing but helps the system retain more information about past events. Figure 5b,c demonstrates the performance of the reservoir in solving NARMA tasks of varying orders with different sampling rates. As can be seen, both fast and slow sampling rates result in larger errors. Therefore, proper selection of sampling frequency is essential in achieving optimal performance from the reservoir in terms of its accuracy.
The performance of a contextual computer while processing temporal data can be quantified through a concept known as MC, defined as the ability of a processor to retrieve past information from the reservoir using the linear combinations of its internal states. Figure 5d demonstrates how the MC of the reservoir is affected by the sampling frequency when solving a NARMA3 task. As can be seen, a slow sampling rate results in the computer operating with the least error (because of the increased nonlinearity) but also a small MC (%2) because the system forgets about past events. On the other hand, a high sampling rate results in   www.advancedsciencenews.com www.advintellsyst.com poor accuracy, but the system remembers many past events (%12). For this reservoir, a balance may be struck by choosing a sampling frequency around 6/TTC to achieve good computation accuracy with a high MC (%6). In general, the optimal performance of physical computers depends on the combination of network complexity, sampling rate, and nonlinearity. The computers need to be designed to attain a certain level of performance through their physical design and optimizing dimensions, number of neurons, types of materials, and other parameters that affect their nonlinearity and time responses.
To demonstrate the need for memory integrated within the reservoir, we compared the performance of the 3D-printed processor with two representative regular feedforward neural network (FNN) constructed in MATLAB, which lacks the ability to store and retain temporal information. The results ( Figure 6) demonstrate that the 3D-printed processor outperformed the FNN, which consisted of three hidden layers on neurons with a sigmoid activation function. Two FNNs with 18 and 50 neurons were built, trained, and compared with the 3D-printed computer. Increasing the number of neurons in the FNN marginally improves its performance, but it cannot match that of the 3Dprinted processor. Even though further optimization of the FNNs with regard to their number of neurons, activation functions, and number of layers is likely to improve performance, it further brings up the possibility of overfitting to the data. It can further be argued that the performance of the 3D-printed computer could also be improved by optimizing it according to the task requirements. Nonetheless, the results emphasize the importance of integrated memory in processing events.

Near-Sensor Data Processing
The 3D-printed computer was employed to process data from a wearable device to detect the type of user activity as a demonstration of its real-life utility. In this case, the data from a 3-axis accelerometer on a wearable device worn on the wrist of a user during different activities were collected. There is a strong correlation between the acceleration data from the three different axes (a x , a y , a z ). To simplify the problem, the acceleration data from the three axes were combined to produce an equivalent signal This signal was then used to extract features provided to the 3D-printed computer for activity detection.
The incoming data were studied in 5 s time frames. The mean, standard deviation, root mean squared value, and the number of peaks of a eq in each time frame were calculated and normalized. These features were then converted into analogue voltage signals and applied to the 3D-printed computer as features of user activity data in that time frame. The training was based on 196 instances of labeled user activity. The weights of the output layer were determined through linear regression. The performance of the computer was evaluated using 60 additional Figure 5. Assessment of the computer performance to solve computational tasks with varying orders of nonlinearity and memory. a) The reservoir response to predicting a time-series input produced by varying orders of NARMA task with f S ¼ f S3 . b,c) Comparison of the reservoir performance when solving NARMA tasks of different orders with different sampling frequencies. d) Demonstration of the dependence of reservoir memory capacity, RMSE, and R 2 to sampling frequency when solving a NARMA5 task. The graph at the top shows a typical step response of one of the nonlinear neurons in the reservoir which is used to estimate TTC and set sampling frequencies.
www.advancedsciencenews.com www.advintellsyst.com instances of labeled activities that were excluded from the training dataset (i.e., validation set). The results of this experiment are shown in Figure 7. Figure 7a shows a sample of raw data collected from the accelerometers (a eq ) as well as features extracted from 5 s timeframes for different activities. Figure 7b shows the performance of the 3D-printed computer in detecting patterns in the data after training. As seen, the processor performs the data processing with 93.3% accuracy, and 92.5%, 92.8%, and 100% sensitivity in determining whether the user is in an elevator, stationary, or walking, respectively. Note that the sampling period, in this case, is 5 s, corresponding to a sampling frequency of about 12/TTC. This level of performance for such a limited set of input features is on par with the performance of typical machine learning algorithms on the same data.

Conclusion
We demonstrated the first 3D-printed physical computer that was built based on the concept of RC. In an RC, the responses of nonlinear, coupled physical elements are used to create  www.advancedsciencenews.com www.advintellsyst.com context from incoming time-series data. The earlier work by several research groups had demonstrated the possibility of using reservoir computers based on photonic components, memristors, and many other physical systems. However, most of the reported physical RCs require sophisticated measurement systems or highly specialized manufacturing processes that often counteract the benefits offered by the approach. We demonstrated the computational capabilities of our printed computer using standard tasks and also used it to solve a real-life problem. 3D-printed computers based on our reported approach can be designed and fabricated in any facility with access to regular 3D printers that can print with both conductive and insulating filaments. Even in the current first-attempt form, numerous ways exist to increase the computer's computational capability without significant practical barriers. Sample approaches include modulating control signals, increasing the number of printed neurons, increasing the number of layers, adjusting the lateral and vertical distances between the neurons (and hence their coupling), and varying trace widths. An optimal design of similar computers should seek a balance between the aforementioned parameters.
Additionally, the presented work paves the way for the realization of fully 3D-printed intelligent systems that combine 3D-printed sensors with a 3D-printed contextual computer to instantly extract context from environmental changes. Furthermore, the principles may be applied to other manufacturing techniques, for instance, by using coupled temperaturesensitive resistors on silicon chips to process data from micromachined sensors and create context without digital computations.

Experimental Section
3D Printing Setup: FDM is the most widely used 3D printing technology. It uses a filament spool fed to an extrusion head with a heated nozzle. Once the extrusion head heats, it softens and lays down the heated material at set locations, where it cools to create a material layer. The nozzle then moves down to deposit the next layer. We used a double extrusion 3D printer (Ultimaker 3) to print the reservoir. [32] The material is printed with a default nozzle size of 0.4 mm. The printer settings were adjusted to have a layer height of 0.1 mm for each printed layer and a wall thickness of 0.8 mm. Thus, the minimum feature size for the conductive trace is 1.6 mm in the x and y directions, while it is 0.1 mm in the z-direction.
The key strength of FDM is the availability of a wide range of materials, including thermoplastics such as PLA, a vegetable-based, biodegradable thermoplastic. PLA is an electrical insulator, so it was used as the substrate and insulating material between the conductive layers. Conductive traces were printed using carbon-PLA composite filaments. We used 2.85 mm pure PLA filament (Ultrafuse series from BASF) for the main structure and 2.85 mm carbon-PLA composite (RM-PL0100 from Lulzbot) for the conductive segments. Before 3D printing, the conductive PLA filament has a volume resistivity of 15 Ω cm. A 3D-printed structure printed in the x-or y-directions has a resistivity of 30 Ω cm, whereas 3D-printed layers printed in the z-direction have a resistivity of 115 Ω cm. Multilayer resistors offer higher conductivity as more conduction paths will be possible due to the increased connections between carbon elements within this structure. Silver conductive paste/epoxy was used to attach wires to the printed contacts, which were cured on a hot plate at 50°C for 1 h.
The lateral dimensions of the 3D-printed reservoir are 3.9 cm Â 3.5 cm, while its height depends on the thickness of the traces (i.e., the number of printed layers). Conductive trace thicknesses vary from 0.5 to 2 mm, the width of the conductive traces is 1.6 mm (twice the minimum feature size), and the gap between the traces (layers) is set to 0.2 mm. Data Collection Setup: For I-V characteristics, we swept the current passing through the NTC forward and backward and measured its voltage after various time intervals by Keysight 2901A precision source/measure unit (SMU). The input signals and output data were collected using a National Instruments PXIe-6363 I/O module through a LabVIEW interface at a rate of one sample per second. The data for user activity detection were collected using the SensorTile kit from STMicroelectronics, which includes a pair of microcontrollers, a 3-axis accelerometer, and other sensors. [33] Training and test data were collected by attaching the module to a user's wrist and recording data during different activities.
The input layer uses an analogue shift register to apply the signal and its past three samples (i.e., uðnÞ, uðn À 1Þ, uðn À 2Þ, and uðn À 3Þ) to the reservoir. The readout layer runs a linear regression on the six signals acquired from the reservoir in MATLAB.
NARMA Task: The nonlinear autoregressive moving average (NARMA) is a discrete-time temporal task with an n th -order time lag. The NARMA time series is given by [34] yðtÞ ¼ α yðt À 1Þ þ β yðt À 1Þ X n i¼1 yðt À iÞ þ γ uðt À nÞuðt À 1Þ þ δ where α ¼ 0.3, β ¼ 0.05, γ ¼ 1.5, and δ ¼ 0.1. The dependence of NARMA on its nonlinearity and long time lags makes it a challenging problem for any computational system. Calculating NARMAn task requires a device capable of algorithmic programming and perfect memory of the input and the outputs of up to n previous time steps. MC: The short-term MC of the RC system is important in applications requiring memory of past events. To evaluate the short-term MC, the τ-delay MC (MCðτÞ) is calculated from [34] MCðτÞ ¼ Cov 2 ðuðt À τÞ, y τ ðtÞÞ σ 2 ðuðtÞÞ⋅σ 2 ðy τ ðtÞÞ where uðt-τÞ is a τ-step delayed input, y τ ðtÞ ¼ uðt-τÞ is its reconstruction at the output, Covð⋅, ⋅Þ is the covariance of the two time-series signals, and σ 2 ð⋅Þ is the variance of a signal. The overall short-term MC is then approximated as

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.