Actively Reconfigurable Segmented Spatial Sound Modulators

High‐quality acousto‐holographic patterns and images, integral to applications like 3D displays, acoustophoresis, and midair haptics, require precise distribution of ultrasound waves to achieve. Essential tools for this task are spatial sound modulators (SSMs), which control constituent elements to enable dynamic distribution of sound pressure. However, current ultrasonic SSMs face limitations due to high costs and the intricate actuation of numerous small, closely spaced units. This study introduces “segmented SSMs,” novel devices that combine traditional acoustic metasurface pixel units into custom‐shaped segmented elements. These segmented SSMs reduce actuation costs and complexity while retaining pressure distribution quality. This approach includes a custom phase agglomeration algorithm (PAA), that offers a hierarchy of potential segmentation solutions for user selection. An SSM fabrication method is detailed using off‐the‐shelf 3D printers and bespoke control electronics, completing an end‐to‐end methodology from conception to realization. This approach is validated with two prototype SSM devices that focus sound waves and levitate polystyrene beads using dynamic segmented elements. Further enhancements to the technique are explored through hybrid SSM devices with both static and dynamic elements. The pipeline facilitates efficient SSM construction across diverse applications and invites the inception of future devices with varying sizes, uses, and actuation mechanisms.


Introduction
Spatial control and manipulation of ultrasonic waves has found wide application in acoustic levitation, [1,2] mid-air haptics, [3] and various industrial and medical fields. [4,5]In all these applications, custom-shaped ultrasound fields are generated by spatial sound modulators (SSMs), which are devices of multiple elements with DOI: 10.1002/admt.202301459reconfigurable phase (and/or amplitude) responses.State-of-the-art techniques for building SSMs include phased arrays of ultrasonic transducers [6] and acoustic metasurfaces. [7]n phased arrays, transducers are driven by signals whose temporal shift and duty cycle determine their phase and amplitude, respectively. [8]he use of many transducers that can be quickly updated (e.g., 40 kHz) makes phased arrays powerful and highly reconfigurable tools for generation of custom pressure distributions.However, these features also lead to substantial construction costs, electronic complexity, high power consumption, and debilitating aliasing effects due to greater-than-wavelength element sizes. [9]coustic metasurfaces offer an alternative method for constructing SSMs that overcomes some of the limitations of phased arrays.They typically employ a single wave source and locally control the phase of the wavefront via the geometrical properties of their reflective, transmissive, or absorbing elements. [10]or example, reflective metasurfaces use relative differences in heights between elements to modulate ultrasonic waves bounced from their surface. [11]Application of the thin lens approximation allows reflective metasurfaces to be modeled quite simply as a continuous surface lying on a two dimensional plane. [12]Metasurfaces are cheap to construct using off-the-shelf 3D printers, and comprise arrays of subwavelength unit cell elements, which generate pressure distributions without aliasing effects. [13,14]However, dynamic reconfiguration of the many subwavelength elements has been a major challenge for the adoption of metasurfaces as a viable alternative to phased arrays in SSM construction. [15]ur primary contribution is a design and fabrication pipeline for constructing segmented SSMs which combines the reconfigurability of phased arrays with the aliasing-free pressure distribution capabilities of acoustic metasurfaces.We still employ cost-effective, pixel-like metasurface unit cells as building blocks for our devices Figure 1a, but we agglomerate them into segments that are controlled by a single parameter (Figure1b), namely the displacement of a mechanical pin.Different segment displacements within the SSM give rise to different Here, many actuators would be required to configure between states.b) We apply our agglomeration algorithm to generate segmented phase maps, which are capable of achieving a desired set of states with far fewer actuators.c) Actuating the segments into different positions allows a segmented SSM to generate a set of three target pressure distributions.d) In the fabrication step, we connect the layer of segmented elements to a custom motorized pin array and control electronics which allows them to move to different heights and provide different responses to incoming waves.
output pressure distributions, as in Figure 1c.We exploit the similarities between finite sets of desired distributions to generate efficient segmented elements.These elements, when taken as a full segmented SSM entity, provide dynamic acoustic holography and levitation at a greatly reduced cost compared to conventional methods.
In the SSM design step of our pipeline, we employ a custom phase agglomeration algorithm (PAA), which outputs a hierarchy of segmentation structures that trade off between actuation cost and pressure distribution quality.At one end of this hierarchy is the "unsegmented" SSM, produced using standard phase retrieval algorithms.Here, each element can have any phase value, resulting in granular phase maps that produce benchmark (maximal) quality pressure distributions.However, this approach incurs maximum actuation costs, as each pixel unit requires its own mechanical actuator.Our methodology reduces actuation cost by clustering the pixel units into contiguous segmented elements, allowing multiple units to share a single actuator.The segments can be moved in unison to generate multiple phase responses and contribute to the formation of the various target pressure distributions.Judging the quality of a segment is a non-trivial process, as traditional metrics are inadequate or computationally prohibitive.The pressure distributions from an SSM are not directly coincident with its surface phase distributions, but are instead linked through a complex directivity relation that describes the interference behavior of sound waves as they travel from the SSM surface to the far field.Consequently, we introduce a custom segment quality metric that accounts for this and assesses the inclusion of viable potential segments into a segmentation structure.The PAA provides a hierarchy of segmentation structures that span a range of actuation costs, allowing users to se-lect the solution that best suits their specific requirements and design constraints.
In the fabrication step of our pipeline, we use consumer and industrial-grade 3D printers to physically realize the segmentation structures output by the PAA into segmented SSM devices.The fabricated segments include pixel units of fixed heights, which locally modulate incident waves at a subwavelength scale in spite of their common actuation, leading to high quality acoustic fields.We use a custom pin-array system to connect actuating stepper motors with the centroids of the irregularly spaced the segments in the structure.Traditional dense and equidistantly arranged pin arrays, [16,17] commonly used in dot matrix or shape display devices, often incorporate elements that do not need precise positioning to fulfill their user interaction requirements. [18]owever, this poses a challenge for current metasurfaces, as their elements rely on accurate positioning for effective phase modulation.Our design avoids this problem by having a much lower actuator density and a much greater precision in the positioning of segmented elements than conventional approaches.This translates to SSMs with greater freedom of actuator placement, lower electronic complexity, and the ability for each segment to generate a precise set of phase responses thanks to the stepper motors attached to each pin in the array.Finally, we designed a custom printed circuit board (PCB) to facilitate control of the stepper motors via an array of drivers.This enables individual step control for each motor to accurately position segments as required to generate the set of desired pressure distributions.A photograph of these components is shown in Figure 1d.
We demonstrate the effectiveness of our design and fabrication process by creating and evaluating ultrasound modulating segmented SSM prototypes.We confirm their effectiveness through simulations and experimentation.We also investigate enhancements to our pipeline by incorporating static elements into our segmented SSM designs to achieve further cost reduction.Our first prototype is an SSM which is able to focus sound waves at five unique positions in space.Focusing of ultrasound is crucial for sound-based haptics and levitation devices, [1] and the ability to adjust the focus position is a particularly desirable and significant feature.We show that an unsegmented SSM of 144 pixel elements can be reconstituted into a segmented SSM of only seven segmented elements, while maintaining the ability to focus sound waves at each of the five locations.Second, we manufactured and tested an ultrasonic levitation device.Ultrasonic acoustic levitation has significant implications in various fields, such as material science and drug discovery, as it allows for non-contact manipulation and study of small objects. [19]Additionally, it offers the potential to create floating, 3D visuals without the need for physical support, revolutionizing our interaction and perception of digital content. [8]The device we constructed features 256 pixel elements reconstituted into 16 segmented elements, and is able to levitate two small polystyrene particles at 12 unique positions.Lastly, we explore even more effective ways to construct SSMs by combining static and dynamic segmented elements together in a single device.Static elements contribute to output pressure distributions without requiring actuators to reconfigure them.The method involves identifying viable dynamic elements for conversion into static ones.This approach benefits applications like acoustic levitation and holographic displays by offering improved performance, manufacturing ease, and cost reduction through static-dynamic hybrid SSMs.
Our work provides three primary contributions: a software pipeline for efficient segmented SSM design, a method for translating these designs into hardware, and validations of the effectiveness of this approach through fabrication, simulations, and experiments.This development paves the way for the creation of affordable SSM devices with low operational complexity, striking a tunable balance between reconfigurability and accurate generation of ultrasonic pressure distributions.

Software for Design of Segmented SSMs
An SSM surface is composed of S unit cell pixels that locally modify the phase of an incident wavefront w ∈ ℂ S generated by an external source.Adjusting the phase  of each pixel s ∈ {1, …, S} results in a modulated wavefront w ⋅ e i over the surface.A distribution of phase adjustments over an SSM surface, gives rise to custom acoustic pressure distributions at an arbitrary set of points p ∈ {1, …, P} in the far field: where  p, s describes the acoustic transmission following some directivity relation (e.g., Angular Spectrum Method) from a unit cell s to some point p.A primary feature of SSMs is their ability to reconfigure themselves to produce multiple phase responses  ∈ ℝ S×H .These responses correspond to multiple unique pressure distributions  ∈ ℂ P×H , where h ∈ {1, …, H} are the different configurations or "modes" of the SSM.So far, generation of  has been achieved using standard "unsegmented" phase distributions  found using off-the-shelf phase retrieval algorithms.Achieving reconfiguration between different SSM modes using a single unsegmented device is currently challenging due to the high cost and engineering complexity associated with actuating each individual pixel unit.
In this work, we propose a custom phase agglomeration algorithm to generate alternative segmented phase distributions Φ ∈ ℝ S×H .Here, we agglomerate groups of neighboring unit pixels to form "segmented elements" that can be actuated in unison.In Figure 2a, two sets of unit pixels have been selected for agglomeration into a single, larger segment.The S c agglomerated pixels within each segment c ∈ {1, …, C} can be identified by the indices I c ⊂{1, …, S} in Φ. [20] These pixels have fixed phase displacements  c ∈ ℝ S c (i.e., they are static relative to one another), but are reconfigured from mode to mode by some constant phase actuation applied in unison to all pixels in the segment  h c ∈ ℝ S c (Figure 2b).The phase distributions Φ are comprised by segment submatrices Φc ∈ ℝ S c ×H , whose columns differ by the constant phase actuation vectors  h c (Figure 2c,d), but which still give rise to pressure distributions Ψ that closely approximate the target distributions , as illustrated in Figure 2e.

Phase Agglomeration Algorithm
Our custom PAA generates a hierarchy of SSM phase distributions for a given segmentation problem (a given set of target pressure distributions ).At each stage of this hierarchy, we iterate over the total number of segments C = (S, S − 1, …, 1) making up the the SSM.In each iteration, the pixels of two segments are agglomerated into one larger segment, enabling the total number of mechanical actuators to be reduced by one.This is illustrated in Figure 2 as the two segments highlighted in Figure 2a are agglomerated into a single segment in Figure 2d.
Each ΦC ∈ ℝ S×H in the hierarchy gives rise to a set of H output pressure fields.The first phase distribution in any instance of the PAA ΦC=S is commensurate with , in the sense that each of the S pixels is considered a segment in its own right.We generate these "unsegmented" phase distributions through the solution of standard phase retrieval problems for a given set of target pressure distributions (see Section SB Supporting Information).From here, the PAA outputs a hierarchical set of ΦC containing progressively fewer segments as the hierarchy is traversed, such that the final SSM ΦC=1 consists of a single segment containing all the pixels.
The key challenge for our PAA was to select which pair of segments will be agglomerated at each iteration.To this end, we introduce a "contiguous combinations" algorithm that restricts the consideration of candidate segments to those with adjacently connected pixels.This ensures that agglomerated segments can share a mechanical actuator, but also reduces the (extremely large) search space for candidate pairs of segments.Following this, we select the pairs of segments via a custom metric that assesses the level of similarity between the segment phases across all SSM modes.That is, the similarity between the columns of Φc ∈ ℝ S c ×H .In the hierarchy of solutions generated by the PAA, a reduction in the number of segments (i.e., number of actuators) tends to result in the reduction of the pressure distribution quality Ψ. Knowledge of the relationship between these two crucial factors enables users to make informed decisions when choosing a segmented SSM design to physicalize.By carefully selecting the solution that strikes the most favorable balance between number of segments and pressure distribution quality, users can align the design with their specific requirements and limitations.

Segment Selection
At each iteration, the PAA selects and agglomerates two segments, such that the total number of segments is reduced by one as the algorithm proceeds to the next iteration.To achieve this, the PAA generates a pool of candidate segment pairings (from here on, just "candidate segments") populated from the previous iteration by pairings with at least some contiguously connected pixels.The ideal candidate segments will provide phase responses that contribute to the formation of each of the H target distributions .
To satisfy the physical proximity constraint during agglomeration, we introduce a "contiguous combinations" algorithm.It iterates through segment pairs in the current PAA iteration C. For each pair, an adjacency check is conducted on all pixels in both segments to verify proximity.If adjacent, the pairing is added to the list of valid contiguous segment pairs.This process is repeated for all segments, yielding a list of eligible candidate segments.This reduces the potentially vast pool of candidate segments by eliminating any with no adjacently connected pixels.See Section SC.1 (Supporting Information) for more details.The contiguousness constraint ensures that all candidate segments can be actuated in unison by a single shared actuator.To our knowledge, this approach is unique in the context of hardware problems.Traditional clustering algorithms typically handle abstract, discrete digital objects rather than physical and proximate ones, [21] making cluster selection in these cases more straightforward.
We next need a method for assessing the quality of candidate segments.However, conventional quality metrics such as mean squared error or peak signal-to-noise ratio are inadequate for evaluating the quality of individual SSM segment phase values during the segmentation process.Such metrics could be employed to evaluate pressure distributions output by the full SSM entity (e.g., using the Angular Spectrum Method), repeated for every candidate segment in every PAA iteration, but this is a computationally prohibitive task.A further difficulty arises because the pressure distributions output by an SSM do not directly correspond to phase distributions on its surface, but instead are linked by a complex directivity relation that describes the evolution and interference of sound waves as they propagate from the SSM surface to a far field plane.The degree of contribution from an individual segment therefore remains unknown until the segmentation process is complete and the character of all segments making up the SSM is identified.
To tackle these challenges, we introduce a custom metric that computes the quality of a candidate segment in isolation (i.e., without considering its combined response with other segments on the SSM surface) while the segmentation process is still ongoing.We use a circular distance averaging operation to consider the phase response φh c across all H modes, with equal weighting given to each mode.In this evaluation, we consider the pressure distributions , and the unsegmented phase matrix , which gives rise to them to be of benchmark quality, but at the price of incurring maximum actuation cost.Achieving complete alignment between our segmented elements Φc and the corresponding pixels in  is extremely challenging due to the collective actuations applied to their constituent pixels during mode switching.In a good candidate segment, the pixel phase matrix Φc will exhibit minimal variation across its rows (translating to near-constant changes between columns), while closely approximating the benchmark phase distributions.To this end, our custom metric tracks the directionality and consistency of pixel phase change in a candidate segment across all modes: here,  s h c is the phase displacement (or circular phase difference defined in the range [ − , ]) of each pixel s ∈ I c in segment c and mode h.We find this using the unsegmented phase value  s,h c with the first mode h = 1 employed as a reference: Furthermore, ( s h c ) is the circular mean [22] of phases across all pixels s at each mode h in the segment c: The quality metric Q(c) returns a high score when the phase displacement of pixels s in the segment c and each mode h are not far from their circular mean, i.e., when phase changes across modes are closer to being constant.We evaluate all candidate segments in the pool to return a Q(c) for each.The segment with the highest score is then chosen for agglomeration, resulting in a reduction of one segment from the total number comprising the SSM.The hierarchical nature of the PAA allows the pool of candidate segments, indexed by their quality, to persist between iterations.Updates are applied only when a new segment is agglomerated, meaning that recalculation of the vast majority of segments is not required from iteration to iteration.
Having selected the candidate segment c to be agglomerated, in the final stage of the PAA we must find the phase responses φh c it will provide in each mode.Due to the circular distance averaging operations applied to the segment that allow its constituent pixels to share an actuator, the segmented phase response will not be fully commensurate with its unsegmented equivalent  h c .Instead, φh c is composed of a phase displacement vector  c and phase actuation vectors [ 1 c , … ,  H c ] for each of the H modes.The vector  c ∈ ℝ S c corresponds to the fixed phase values that will be physicalized as height differences on the segment surface and are therefore constant in all modes and is found for each segment pixel s ∈ I c and mode h as: The vector  h c ∈ ℝ S c is constant (i.e., all the phase quantities contained within it are equal) and is imparted to the segment by its mechanical actuator as the device is reconfigured.Its value is given by the circular mean of unsegmented phase values across the same pixels: At any given h, the total phase response φh c of segment c is given by the summation of these two vectors φh c =  c +  h c .The corresponding column of the segmented phase matrix Φ can now be updated with these new phase values and this iteration of the PAA is complete.As the PAA continues to iterate and the number of segments C decreases, we progressively generate a hierarchy of segmentation solutions.

Hardware Realization
In this section, we describe the physical realization of segmented SSMs designed using our PAA algorithm.We begin by selecting a segmentation structure from the hierarchy generated by the PAA.This selection hinges is based on achieving an optimized balance between actuation cost and the quality of output pressure distributions tailored to the specific user of our pipeline.In Figure 3a, we plot the segmentation hierarchy as a comparison between the number of segments making up a given segmentation structure C and its quality Y(C).This quality is measured as the similarity between its output pressure distributions and those of its benchmark unsegmented equivalent.We highlight a segmentation structure with C = 15 and Y(C) = 0.858 as an example of a selection made with specific user requirements in mind.User requirements could include an upper bound on device cost, limited space for actuator placements, or a minimum level of accuracy for the output pressure distributions.Following the user selection, we physicalize the phase displacements  c and actuations  h c of each segment by converting them into height displacements d c and actuations a h c .We use a custom pin array system and printed circuit board (PCB) to provide actuation to the physicalized segments, allowing the SSM to switch between modes, as illustrated in Figure 3b.
To realize each mode of the SSM device and achieve the correct phase responses, precise positioning of the segments is essential.Every component of the physical SSM system must function precisely as a whole, free from frictional hindrance.Our 3Dprinted reflective segments are designed to move smoothly past each other and through the guiding frame with minimal friction.Additionally, our custom pin array system rigidly connects to the centroids of the segments at one end, while detachable friction clips secure it to a stepper motor at the other end.This setup enables accurate movement and maintenance of segments while allowing disassembly for reuse of the PCB controller.This controller drives the motors and stores positional information for various SSM states, ensuring that motor steps are precisely tracked and translated into segment positions.This comprehensive system guarantees the precise execution of each SSM mode and the manifestation of the desired phase responses by the SSM device.

Selecting Segmentation Solutions for Fabrication
In contrast to other clustering techniques like k-means [23] or density-based spatial clustering, [24] our PAA offers a diverse range of SSM segmentations.This flexibility empowers users to tailor devices they fabricate according to their own unique constraints and requirements.The PAA starts from the most granular segmentation pattern (C = S), representing an "unsegmented" SSM with benchmark-quality pressure distributions.It then progresses to C = 1, where all pixels are agglomerated into one large segment.In doing this, it generates a hierarchy of segmented phase maps ΦC for all possible numbers of segments C = (S, S − 1, …, 1).This comprehensive output covers the entire spectrum of potential actuator costs.
As a result, for each segmentation structure in this hierarchy, we evaluate the quality of (reflected) acoustic distributions ΨC as an average across each mode h.The solution with the most favorable trade-off between actuation cost and pressure distribution quality, is then selected by the user, as shown in Figure 3a.We evaluate the similarity Y(C) of the SSM output pressure distributions for each segmentation solution, using the structural similarity index measure (SSIM). [25]This metric assess the similarity between the segmented and benchmark acoustic pressure distributions as: Simple pixelwise metrics, such as mean squared error or peak signal-to-noise ratio, may not correlate well with perceived image quality as they prioritize accuracy of intensity from pixel-to-pixel, without capturing properties inherent to the image as a whole.In contrast to these simpler metrics, SSIM is a more sophisticated approach that evaluates the structural and perceptual aspects of two images with negligible added computational cost.In situations where evaluating image quality relies on perceived similarity, like mid-air haptics [26] or acoustic holographic symbols and signage, [27] SSIM is a more suitable and efficient choice of metric.Its ability to encompass both structural similarities and pixel intensity differences aligns well with human perception of image quality.SSIM also proves valuable in scenarios where the acoustic pressure distributions whose similarity is being compared are not consistently aligned across both images.This misalignment can occur during physical pressure field measurements, where slight discrepancies between pixels arise due to experimental errors.In such cases, basic pixelwise metrics become less effective and the structural considerations of SSIM make it a more reliable choice of metric.

Physicalizing a Segmented SSM
In the final stage of our pipeline, we physicalize the segmentation structure selected by the user of the PAA.We denote these as Φ⋆ ∈ ℝ S×H , comprised by the phase submatrices Φ⋆ c ∈ ℝ S c ×H of each segment c ∈ {1, …, C}.As described in Section 2.2, these are formed by a displacement vector  ⋆ c and set of actuation vectors  h⋆ c for each mode h.
To achieve the desired phase modulation, for each segment we convert these phase vectors of unit pixel phases into corresponding physical height vectors.The phase displacement or actuation vectors of a segment can be changed into height displacement d ⋆ c or actuation vectors a h⋆ c as: where  is the wavelength of the operating frequency of the SSM.This translation process converts the overall ( ⋆ c +  h⋆ c ) phase response of each segment in each mode into modulations of the path lengths of waves as they are reflected from the faces of the segments.
Following this translation, we proceed to actualize the segments using off-the-shelf computer-aided design software and 3D printers to create rigid structures with reflective topside faces, made up of contiguously connected pixels with displacements defined by d ⋆ c .We design the segments with an air gap of 0.3mm between neighbors, so that they do not experience friction during reconfiguration.The true gap is slightly less than this due to the tolerances of the 3D printer, but is still sufficient to facilitate friction-free movement.An example set of reflective segmented elements is shown at the top the SSM illustrated in Figure 3b.We connect the undersides of these segments to a custom, heterogeneous pin array system, uniquely designed for each segmentation structure.The pins take the form of squared pillars connected on their top sides to the centroids on the underside of each reflective element.We slot them into a frame, providing a channel of vertical movement for the segment, along with a number of guide rails, which prevent the segment from twisting in unintended directions.
We connect the bottom sides of the pins to the set of actuators.The segmented SSMs described here feature simple stepper motor actuators.However, equivalent actuation systems such as electrohydraulic [28] or solenoids [29] could also be employed effectively in our framework.Our stepper motors are planetary geared, have a full movement range of 10mm, and a deep sub millimeter step size of 0.01mm.The motors control the segment positions using a lead screw drive shaft.We connect the stems of each reflective element to this drive shaft using a custom designed "friction clip".This design facilitates a secure connection between the segments and motors through friction alone and ensures precise control of segment positions by the motors while also allowing for easy detachment during disassembly.A labeled diagram showing an example version of the pin array system is shown in the middle part of Figure 3b.The inset here shows a more detailed view of how a pin in the array is connected to the stepper motor actuator below and the reflector above.
Finally, we achieve control over the stepper motors using a PCB controller of our own design that sends actuations a h⋆ c to each motor.Drivers on the PCB move the segments to the correct positions in each mode to reconfigure the reflective surface of the SSM.Commands are issued from a computer connected to the PCB via serial communication or directly through a set of buttons and an encoder dial.The buttons allow us to cycle through preset actuation vectors stored in the microcontroller, as well as providing the ability to isolate motors for individual tuning via the encoder dial.Commands are issued to the motor drivers in units of motor steps.A positive number moves a segment upwards and a negative number moves it downwards.The PCB used in our designs features 16 motor drivers, [30] and therefore puts an upper bound on the number of available motors (and segments) for SSM designs we physicalize.Each driver is linked to its respective stepper motor via connectors and a short wire.All of this allows per-motor individual step control, and the ability to push and pull any segment into the correct position for each mode.A labeled diagram of the PCB is shown at the bottom of Figure 3b.

Multi-Position Focusing SSM
We showcase the effectiveness of our pipeline through the construction and evaluation of a segmented SSM prototype with five distinct modes.Our device utilizes a single transducer andfocuses ultrasound waves to five different points in space, employing a reflective surface comprised of a 12 × 12 grid of pixels.Figure 4a depicts the complete segmented SSM device, including the transducer source, reflecting, and focusing sound waves to a point in the far field.
The focusing of sound waves is a critical function of sound modulating devices across various fields.Precision focusing underpins many of the most common usages for SSMs including acoustic levitation, [19,31] mid-air haptics, [32] high intensity focused ultrasound for medical applications, [33,34] and even acoustic energy harvesting. [35]Integrating an SSM into applications in these domains facilitates easy focus repositioning.This enhances the adaptability and versatility of these systems and enables the generation of dynamic and personalized sound fields.In our prototype system, an external 40kHz transducer source emits waves toward the center-point of the SSM at coordinates (x = 0, y = 0, z = 0).The source lies at (x = −4.71mm,y = 0, z = 8.58mm) and is driven by a function generator with a sine wave signal of amplitude 20V peak-to-peak.We designed the SSM to focus waves at positions on the far side of the device from the source, maintaining y = 0 and varying the z coordinate as the device transitions between modes.The SSM surface consists of 12 × 12 unit pixels populating seven segmented elements.Each pixel is square shaped, with side length 4.2875mm (/2), resulting in a total surface area for the device of 2647.1mm 2 .This approach prevents occlusion of the incident sound pressure by the measuring device prior to reaching the SSM surface.
We employ a pressure field microphone (B&K 4138-a-015) with a dynamic range of 52.2 to 168 dB (0.008 to 5024Pa).The microphone is secured to a three-axis computer numerical control chassis for precise measurement of the acoustic pressure fields emitted at each of the five SSM modes.This enables meticulous positioning in all three axes.We scan the microphone in an x-z plane measuring (75mm × 120mm) to capture the complex acoustic pressure Ψ reflected from the SSM surface.Our approach combines absolute pressure and phase measurements to determine the complex pressure.The phase measurements are provided by referencing a second channel of the function generator that outputs a constant, known signal.Additionally, source waves originating directly from the transducer (not reflected by the SSM) must be considered.To account for their presence in the measurement plane, we record these waves in isolation (i.e., when there is no SSM present to create reflections) and subtract their complex pressure from the separately measured reflected field.This allows us to show the reflected field as clearly as possible, highlighting the modulation capabilities of our device.
In Figure 4b, we display the simulated absolute pressure | Ψ| distributions for the five modes of the SSM device, obtained using Equation (1).The corresponding pressure measurements are shown in Figure 4c.As expected, the focal point generated by the device exhibited upward movement along the z-axis as the SSM was reconfigured.The measured data generally follows the trend predicted by the simulations.However, while a portion is removed through subtraction, a certain level of parasitic reflection from experimental equipment and nearby rigid surfaces remains.This subtraction can also give a slight "wobble" effect to the measured pressure distributions.These factors contribute to slight discrepancies between the simulated and measured results.It should be noted that the simulated models do not consider these specific factors, which could account for the observed differences.
An unsegmented SSM device with a naive pin array actuation mechanism would require 144 actuators, leading to high costs and complexity.In contrast, our SSM device achieved comparable pressure distributions using only seven segmented elements.This substantial reduction in actuation cost, amounting to just 5% of the unsegmented value, underscores the power and efficacy of our methodology.
Our research showcases a cost-effective design and fabrication method for an SSM that achieves precise sound wave focusing and reconfigurability.This introduces novel avenues for contactless object manipulation and the creation of tactile sensations in mid-air.Our findings provide researchers and practitioners with a new toolkit for practical and efficient ultrasound focusing for a wide-range of applications.

Acoustic Levitation
With our next prototype, we showcase and discuss the results of a novel acoustic levitation device incorporating a segmented SSM.Acoustic levitation is a technique in which small objects are trapped and maneuvered in mid-air using sound waves. [36]t offers contactless manipulation capabilities with applications in areas such as materials science and pharmaceuticals. [37,38]owever, the cost and complexity of the conventional phased array of ultrasonic transducers that currently dominate levitation systems limits their widespread adoption.In this section, we present a levitation system that incorporates an integrated segmented SSM with a single phased array (rather than the conventional pair of phased arrays).This offers cost-efficient actuation while achieving diffraction-limited pressure distributions.
Our system, consisting of a square arrangement of 256 phased array transducers and a segmented SSM of 16 × 16 unit pixels reconstituted into just 16 segmented elements, enables levitation of two particles at 12 different positions (h = 12) in space.Each transducer has a diameter of 10mm and a spacing of 0.5mm, meaning that the surface area of the phased array is 28224mm 2 .The surface area of the segmented SSM is 4705.96mm 2 .A diagram of the levitation system is shown in Figure 5a.By positioning the SSM parallel and opposite to the phased array transducers, we create traps and perform levitation within the cavity.When our system transitions between modes it reconfigures the phase distributions on the surfaces of both the phased array and the segmented SSM.The particle traps experience simultaneous displacement along the x and y axes, respectively, while maintaining constant (but distinct) heights along the z-axis.This emphasizes the ability of our levitation system to generate multiple traps at diverse locations and to displace these traps along separate axes, effectively demonstrating comprehensive 3D dynamic capabilities.We model our system as a sequence of holograms, [39] enabling us to simultaneously optimize the phase configurations of both the transducer array and the SSM.This optimization aims to maximize the trapping stiffness (i.e., the Laplacian of the Gor'kov potential) at the points of interest. [36]By fine-tuning the interference between the source and reflected pressure in this way, we strategically generate trapping points at the desired positions.
In Figure 5b,c,d(i,ii), simulation results showcase the propagated pressure distributions in xz and yz-planes for different device modes, illustrating incremental (3 and 4) and larger (7) displacements of the particle traps.Figure 5b,c,d(iii) shows photographs of the levitation system successfully demonstrating suspension of two polystyrene particles at various positions.The choice of the trap positions shown in this figure highlights the ability of our system to switch between both small and large increments of distance when the mode is changed.
A shared limitation among all acoustic levitation systems, including conventional phased array-only devices, is that the strength of traps diminishes the further they are from the center (x = 0, y = 0).This is due to the simple fact that sound wave intensity decreases as distance from the source increases, making maintenance of particle suspension more challenging in these regions.In our case, the fact that the waves must travel from the phased array source to the reflective surface of the SSM, and then reflect from here to the trapping location, makes this issue even more salient.In the future, a more advanced segmented SSM with a larger reflective surface and additional dynamic elements could capture more of the incident pressure and generate higher-pressure trapping points in these areas.
The need for phased array modulation in addition to SSM modulation for creation of particle traps stems from the the relative weakness of the pressure reflected from the SSM surface.The pressure landscape in the cavity where levitation is performed is dominated by waves coming directly from the transducer.Consequently, achieving effective trapping using SSM modulation alone remains challenging, even when all source transducers emit waves with identical phases.A more advanced SSM with a larger surface area and more segments has the potential to contribute more pressure, potentially enabling trap generation without requiring modulation from the transducers.In this scenario, the SSM would handle all modulation tasks, simplifying the device and reducing its overall complexity and cost.This could potentially allow for a single, adequately powered transducer to serve as the wave source rather than an intricate and expensive phased array setup.
Our findings pave the way for the development of efficient and adaptable acoustic levitation systems with enhanced trapping capabilities.Our levitation system uniquely combines the diffraction-limited modulation capabilities of an acoustic metasurface with reconfigurability, all while significantly diminishing costs compared to conventional methods.By mitigating the limitations of existing devices in this way and reducing the overhead cost and complexity associated with levitation systems, our approach has the potential to catalyze the widespread adoption of acoustic levitation across diverse domains.This could create novel opportunities in areas such as non-destructive and precise positioning of objects, drug delivery and levitation-based 3D printing.

Static/Dynamic Hybrid SSMs
In this section, we explore the creation of static/dynamic hybrid SSMs through a simulated analysis.Inclusion of static elements in the SSM surface, unconnected to any actuators, enables them to contribute to pressure distribution generation without incurring the actuation cost of a dynamic element.This makes them effectively "free of cost" in terms of hardware requirements.However, determining which elements can be made static and which must remain dynamic to achieve the desired pressure distributions poses a challenge.
To address this, our method for creating hybrid SSMs extends the design pipeline outlined in Section 2. To assess the viability of converting segmented elements in a given segmentation structure of size C from dynamic to static, we apply an additional algorithmic layer after Section 2.2.This is done by calculating the summed phase variance V(c) [22] of each segment in a segmentation structure in each mode: Segments with less total variance between modes are more suitable to be converted from dynamic to static.Performing this calculation for all segments in a structure results in a list of elements indexed by their phase variance viability.
We identify a desired number of dynamic elements D to be present in the hybrid structures we create.As such, by systematically transforming the most viable dynamic elements into static ones, the desired number of exactly D dynamic elements can be achieved.Static elements must present a constant phase response across all SSM modes.We therefore use Equation ( 4) to find an average phase response that can be applied to the segment and kept constant for each mode.The full phase matrices ΦCD ∈ ℝ S×H and resulting complex pressure matrices ΨCD ∈ ℂ P×H of the hybrid SSM are then generated for each mode, as outlined in Section 2.2, incorporating the phases of static elements.
As our PAA outputs a hierarchy of C segmentation solutions, there are C − D possible ways that a hybrid SSM can be constructed with D dynamic elements.Each of these potential hybrid SSMs contain exactly D dynamic elements and are considered to be equal in cost.Therefore, it is logical to determine which phase matrix ΦCD in the interval C ∈ (S, S − 1, …, S − D) offers the highest quality ΨCD and select that structure for actualiza-tion.Pressure distribution quality is calculated using the SSIM metric, as outlined in Section 3.1.Whichever structure is chosen in this way, it will contain exactly D dynamic elements.Fabrication of this segmented SSM would follow the methodology outlined in Section 3.2, but with static elements simply fused to the frame in the required position rather than being connected to an actuator.
In order to evaluate the effectiveness of a hybrid SSM constructed in this way, we perform a simulated analysis comparing them to dynamic-only SSM designs, also containing D dynamic elements.Both the dynamic-only and hybrid SSMs in this analysis contain 32 × 32 unit pixels of diameter /2 and thus have surface areas of 18823.8mm 2 .Figure 6 showcases a pair of example SSMs that generate acoustic holographic images of the letters "U, C & L" parallel to and at a distance of 10 40kHz (85.75mm) from the SSM surface.Figure 6a(i) illustrates a naively segmented, dynamic-only SSM design with D = 256 segmented elements, colored in red.The output pressure distributions and their corresponding SSIM scores are shown in a)(ii, -iv).This naive SSM is constructed of simple segments made of 2 × 2 groupings of pixels.In contrast, Figure 6b showcases an equivalent hybrid SSM design, also containing D = 256 dynamic elements (red), as well as static elements (green).Again, the output pressure distributions and their corresponding SSIM scores are shown in b)(ii-iv).
The hybrid SSM consistently yields higher SSIM scores and more accurate acoustic holographic images than its naively segmented, dynamic-only counterpart by allowing unactuated (and thus "free of cost") static elements to be included in the design.The heterogeneous nature of the hybrid segmentation structure and subsequent ability for many of the elements to remain subwavelength in size results in significantly reduced aliasing effects for the hybrid pressure distributions.Furthermore, a hybrid SSM demonstrates improved performance in comparison to a fully dynamic SSM developed using the methodology described in Sections 2 and 3.Both approaches result in SSMs that retain a good deal of subwavelength structure.The inclusion of static elements at appropriate locations means that the actuators can be deployed in areas of the SSM surface where they are most needed.Thus, a hybrid SSM outputs the same set of sound fields at equal or better quality using substantially fewer actuators, reducing device cost, and complexity compared to a fully dynamic equivalent.The hybrid SSMs developed in this study are particularly beneficial in scenarios where the phase distributions are very similar across modes.Acoustic levitation is a good example of this, in cases where small phase shifts are used to displace particles over small distances.In these situations, segments on the SSM surface that experience minimal phase changes during mode-switching can be made static without significantly impacting the output pressure distributions.
In conclusion, the incorporation of static-dynamic hybrid SSMs presents a promising avenue for efficient SSM design.These hybrid devices offer enhanced image quality compared to dynamic-only SSMs with the same number of D dynamic elements.The ability to include static elements in the design not only improves performance but also facilitates easier manufacturing and reduces costs.Future applications in the fields of acoustic levitation, holographic displays, and other related technologies can benefit from these advantages of hybrid SSMs.

Conclusion
In conclusion, we have presented a novel pipeline for the construction of segmented SSMs that combines the strengths of phased arrays and acoustic metasurfaces, while addressing their respective limitations.We introduce segmented elements, formed by optimized agglomerations of traditional metasurface pixel elements.Our pipeline offers a robust method for SSM design and fabrication, facilitating the development of affordable devices with simplified operation.We validated the pipeline through construction of prototype segmented SSM devices, which we evaluated through experimentation and simulation.
The most significant contribution of this research is the development of a custom phase agglomeration algorithm that generates a hierarchy of segmentation structures.This algorithm reduces the number of active surface elements while preserving high quality output pressure distributions.With the implementation of the PAA, users can design and manufacture SSMs capable of generating specific sets of pressure distributions.This approach offers a cost-effective solution that strikes a balance between quality and reconfigurability.
We construct prototype devices and test their effectiveness through simulations and experiments.We demonstrated the ability of a segmented SSM to focus sound waves at five distinct positions in space, highlighting the transformation of an unsegmented SSM into a segmented one with significantly fewer elements.Additionally, we manufactured an ultrasonic levitation device that showcased the potential of our device to expand the accessibility and adoption of SSM technology within the acoustic levitation community.Finally, our exploration of static/dynamic hybrid SSMs provides valuable insights into further enhance-ments of our methodology by offering even greater reduction in actuation cost.
Looking ahead, future research could involve integration of multifrequency devices into our pipeline, which would broaden the application space by transcending the limited application space of monofrequency devices.Additionally, multiple segmented SSMs working together with strategically positioned voids could provide new opportunities for sound control as a distributed system of SSM devices.Finally, exploring alternative baseline clustering approaches (such as k-means [23] or densitybased spatial clustering [24] ) could offer improved segmentation capabilities of our pipeline, but could also limit the advantages of the hierarchy of solutions output by our agglomerative clustering approach.
Our pipeline for constructing SSM devices offers a novel and efficient methodology that combines the strengths of phased arrays and acoustic metasurfaces.We substantiate our claims through the successful implementation of the pipeline in design and construction of prototype devices.These contributions are of significant value to the spatial sound modulator community, providing cost-effective, reconfigurable, and high-quality solutions.The outlined future pathways present exciting opportunities for advancements, expanding the horizons of spatial sound modulation and its applications across diverse fields.

Figure 1 .
Figure 1.Diagram of our segmented SSM design and fabrication pipeline.a) Three Phase maps A, B, and C (red, blue, and pink, respectively) are encoded into an "unsegmented" surface made up of traditional acoustic metasurface unit pixels.Here, many actuators would be required to configure between states.b) We apply our agglomeration algorithm to generate segmented phase maps, which are capable of achieving a desired set of states with far fewer actuators.c) Actuating the segments into different positions allows a segmented SSM to generate a set of three target pressure distributions.d) In the fabrication step, we connect the layer of segmented elements to a custom motorized pin array and control electronics which allows them to move to different heights and provide different responses to incoming waves.

Figure 2 .
Figure 2. Overview of the steps for creating a segment in one iteration of the PAA.a) Matrix of SSM phase distributions ΦC at iteration C of the PAA, with a pair of contiguous segments highlighted in red.b) These segments are agglomerated to form a new segment c.We calculate the phase displacement vector  c and phase actuation vectors  h c for each mode.c) Summing these vectors returns the segmented phase responses φh c for the newly agglomerated segment in each mode.These are combined into a matrix Φc ∈ ℝ S c ×H .d) The matrices Φc are stacked and recombined with the remainder of ΦC to form ΦC−1 ∈ ℝ S×H , describing the SSM entity at the iteration C − 1. e) The resulting pressure distributions output by the segmented SSM at this iteration are denoted as ΨC−1 ∈ ℂ P×H .

Figure 3 .
Figure 3. a) Plot showing the average SSIM score Y(C) against number of segments C for a full hierarchy of segmentation solutions output by the PAA.b) Diagram displaying and labeling the three main components of a segmented SSM: the reflective segmented elements (top); custom pin array of stepper motors (middle); and PCB controller (bottom).The inset shows a zoomed in view of the actuation system connecting a stepper motor actuator to a reflective element.

Figure 4 .
Figure 4. a) Diagram of the multi-mode SSM device focusing sound to a single point.b) Simulated pressure distributions for the five modes of the device.The focusing position in the z-axis is reported at the top of each plot.c) The measured versions of the same pressure distributions shown in (b).

Figure 5 .
Figure 5. a) Diagram of the SSM based levitation system.The xz levitation plane is shown in red, while the yz levitation plane is shown in blue.b)(i) andii show 2D slices of 3D simulated pressure distributions generated when the particle is displaced by 3 from x = 0 and y = 0 respectively.b)(iii) shows a photograph of the system from the yz-plane perspective levitating a polystyrene particle in this position.in these photographs the pairs of beads are at different depths.c, d) show the same, but for particle displacements of 4 and 7, respectively.

Figure 6 .
Figure 6.Segmentation structures for a) dynamic only and b) hybrid SSMs for generation of the same acoustic holographic images of the letters "U, C, L" are compared.In a)(i) and b)(i) the segmentation structures are illustrated, with static elements and dynamic elements denoted in green and red, respectively.In a, b) (ii-iv), the acoustic pressure distributions output by these hybrid SSMs are shown, along with their corresponding SSIM scores.