In designing three-layer (cladding/core/cladding) waveguide structures that support low-loss surface-bound waves at both interfaces of core layer, a hyperbolic metamaterial can be employed for either a core or cladding layer. From the implementation perspective, however, a metamaterial-core structure is preferred: this is because a metamaterial-cladding structure would require thicker metamaterial, therefore, more metal layers, although the metamaterial-cladding would, in general, allow stronger confinement of light in the lateral direction. In this work we focus on the metamaterial-core case with alternative application potential in mind, that is, to exploit the loosely-bound nature of low-loss surface plasmons. Further we are interested in the waveguide structures that will involve a minimum number of metal layers incurring lowest possible losses.

Figure 7a shows a schematic of a three-layer waveguide structure that employs a metamaterial core sandwiched by dielectric cladding such as SiO_{2} (Q) or soda-lime glass (G). Here the waveguide core part is assumed to consist of Ag/SiO_{2}/Ag three-layer thin-film stack with Ag composition *f*_{Ag} of 0.1, and is modeled as a homogeneous metamaterial possessing a dielectric tensor (*ε*_{
m
}) that was calculated by applying an effective medium approximation at 1.3 µm wavelength: *ε*_{
m,n
} = 2.332 + *i*0.000141 and *ε*_{
m,t
} = − 7.01 + *i*0.2056. Note that this dielectric/metamaterial/dielectric structure corresponds to dielectric mismatch \( \Delta \varepsilon \) of 0.238 or 0.052 for SiO_{2} or soda-lime glass cladding case, respectively. The following equations are solved to calculate propagation length and penetration depth into cladding of symmetric surface-bound mode supported by this three-layer waveguide structure.

$$ \frac{{\varepsilon_{d} }}{{\varepsilon_{m,t} }} + \frac{{\gamma_{d} }}{{\gamma_{m} }} \cdot tanh^{ - 1} \left( {\gamma_{m} \cdot {\raise0.7ex\hbox{$a$} \!\mathord{\left/ {\vphantom {a 2}}\right.\kern-0pt} \!\lower0.7ex\hbox{$2$}}} \right) = 0 $$

(12)

$$ \frac{{\beta^{2} }}{{\varepsilon_{m,n} }} - \frac{{\gamma_{m}^{2} }}{{\varepsilon_{m,t} }} = k_{0}^{2} $$

(13)

$$ \frac{{\beta^{2} }}{{\varepsilon_{d} }} - \frac{{\gamma_{d}^{2} }}{{\varepsilon_{d} }} = k_{0}^{2} $$

(14)

Here, *a* denotes the thickness of metamaterial core layer. Other parameters are the same as above. There can be multiple solutions of this transcendental equation set, but we will focus on the fundamental mode (symmetric and surface-bound), which demonstrates the lowest loss.

Figure 7b, c show the result of analytical calculation of propagation length and penetration depth for core thickness in the range of 10–250 nm. In the better-matched case (i.e., ∆*ɛ* of 0.052 for soda-lime glass cladding; red), 93-mm propagation length is attainable at 200-nm core thickness, while penetration depth into cladding is calculated to be 8.6 µm. In the case of silica cladding (∆*ɛ* of 0.238; blue) propagation length and penetration depth at the same core thickness (200 nm) are estimated to be 9.1 mm and 2.2 µm, respectively. Note that both propagation length and penetration depth monotonically increase as core thickness is reduced. The conventional bulk-metal core case (black dotted) is also shown for comparison: 223-µm propagation length and 845-nm penetration depth at 200-nm core (Ag) thickness.

FDTD analysis was also performed on the original five-layer structure (glass/[Ag/SiO_{2}/Ag]/glass) case, where the metamaterial core part (with *f*_{Ag} = 0.1) is assumed to consist of three layers (10-nm-Ag/180-nm-SiO_{2}/10-nm-Ag) [8]. Figure 8 shows field distributions (*H*_{
z
}, *E*_{
y
} and *E*_{
x
}) calculated at 1.3 µm wavelength. From the *E*_{
y
} field plot (green in Fig. 8a) the field amplitude decays from 0.6 at *y* = 0.1 µm to 0.5 at *y* = 2.0 µm. Assuming an exponential decay profile the ratio of the two amplitudes can be expressed as: \( \frac{{\left| {E_{y1} } \right|}}{{\left| {E_{y2} } \right|}} = \frac{{\left| {E_{y0} } \right|e^{{ - \frac{y1}{L}}} }}{{\left| {E_{y0} } \right|e^{{ - \frac{y2}{L}}} }} \). The penetration depth is then calculated to be:\( L = \frac{y2 - y1}{{{ \ln }\frac{{\left| {E_{y1} } \right|}}{{\left| {E_{y2} } \right|}}}} = \frac{2 - 0.1}{{\ln \frac{0.6}{0.5}}} = 10.4\; \upmu {\text{m}} \). This number shows reasonable agreement with the analytical calculation (8.6 µm at 200-nm core thickness: see Fig. 7c). This comparison again validates the effective medium approximation applied to a well-matched metamaterial/dielectric system with a small number of constituent layers. Both tangential and normal components of *E*-field remain low in metal layers, whereas normal *E*-field maintains its strength in dielectric layers. The normal *E*-field profile (green) demonstrates a highly-localized (into dielectric layers) and yet broad (with large penetration depth) distribution, enabling low-loss propagation of SPs. Note also that normal *E*-field (*E*_{
y
}) takes different signs in metal and dielectric layers: see Fig. 8b, green.

Next we analyzed the simplest (i.e., a single metal layer core) waveguide structure: a thin metal film core is sandwiched by symmetric dielectric cladding [7]. One might view this structure as a special case of the three-layer core waveguide structure discussed above: the thickness of spacer dielectric (SiO_{2}) layer in the core part is reduced to zero under the assumption that the total (combined) metal thickness remains significantly smaller than skin depth (~ 20 nm). The dispersion characteristics in the core layer part, however, significantly differ between the two cases: elliptical for the single metal core case, whereas hyperbolic for the metamaterial core case. In terms of energy flow along the waveguide direction, the Poynting vector (time-averaged energy flow) in the core layer orients to the negative direction (backward) in the metal core case [15]. By contrast, in the metamaterial core case, the energy flow is in the positive direction (forward), the same as that in the cladding layers. The normal dielectric matching condition (∆*ɛ* ~ 0) is no longer applicable to this metal core case, and a symmetric surface-bound wave is always supported regardless of external dielectric constant. The governing equations of this three-layer waveguide structure with a thin-film metal core and symmetric dielectric cladding are given as follows:

$$ \frac{{\varepsilon_{d} }}{{\varepsilon_{m} }} + \frac{{\gamma_{d} }}{{\gamma_{m} }} \cdot tanh^{ - 1} \left( {\gamma_{m} \cdot {\raise0.7ex\hbox{$a$} \!\mathord{\left/ {\vphantom {a 2}}\right.\kern-0pt} \!\lower0.7ex\hbox{$2$}}} \right) = 0 $$

(15)

$$ \frac{{\beta^{2} }}{{\varepsilon_{m} }} - \frac{{\gamma_{m}^{2} }}{{\varepsilon_{m} }} = k_{0}^{2} $$

(16)

$$ \frac{{\beta^{2} }}{{\varepsilon_{d} }} - \frac{{\gamma_{d}^{2} }}{{\varepsilon_{d} }} = k_{0}^{2} $$

(17)

By solving the above equations we calculated propagation length and lateral confinement at 1.3 µm wavelength (Fig. 9a, b). In this calculation the Ag thickness was varied in the range of 10–250 nm and symmetric dielectric cladding is assumed to be silica (Q) or soda-lime glass (G). At 10-nm Ag thickness, propagation length of 52 mm or 63 mm is expected to be attainable for silica (Q) or soda-lime glass (G) cladding case, respectively, with corresponding penetration depth of 3.7 µm (Q) or 4.1 µm (G). As Ag film thickness is increased, both propagation length and penetration depth sharply decrease, asymptotically approaching conventional SPs’ at metal/dielectric interface.

Figure 9c, d show FDTD simulation result of *H*_{
z
}, *E*_{
y
} and *E*_{
x
} field distributions for the case of 10-nm-Ag core with SiO_{2} cladding. Note that normal *E*-field *E*_{
n
} (= *E*_{
y
}) is highly localized to dielectric layers. Also both *E*_{
y
} and *H*_{
z
} fields inside metal remain nearly flat across the metal thickness. This is because the evanescent profiles from both interfaces compensate their decaying profiles, resulting in a nearly constant profile across the metal thickness. As a result of this self-compensation effect, the tangential *E*-field (*E*_{
t
} = *E*_{
x
}) becomes fully suppressed in most of the metal thickness. In other words, referring to the Maxwell’s equation, \( i\gamma_{n} H_{z} = { \mp }\omega \varepsilon_{t} E_{t} \), the tangential *E*-field (*E*_{
t
}) is reduced to zero as the magnetic field (*H*_{
z
}) profile becomes flat (\( \gamma_{n} \sim 0) \). Overall, suppressing the tangential *E*-field results in low-loss propagation of SPs, and this is enabled by employing a thin metal core with symmetric dielectric cladding. Unlike the metamaterial-core case, this thin-metal core structure does not require the condition of good dielectric-matching between core and cladding. Propagation length and penetration depth of a thin-metal core case are a strong function of metal thickness at < 50 nm range, but are less dependent on cladding dielectric constant (Fig. 9a, b). By contrast, in the metamaterial core case, the opposite characteristics (i.e., sensitive to dielectric matching and less sensitive to core layer thickness) are observed, and significantly longer propagation lengths and penetration depths are attainable: e.g., 100–200 mm propagation length and 10–15 µm penetration depth at 120–150 nm core layer thickness with ∆*ɛ* of 0.238 (see Fig. 7b, c). In terms of practicality of implementing the designed structure with good reproducibility (i.e., less prone to process fluctuation such as thickness variation), the metamaterial core structure with a small number of layers offers an advantage over the thin-metal core case.