Update the README for the current figures and results

Fig. 3 is now the 10 dB aggregate with the token-domain bars, the
non-MAML reference carries its figure label, the load-sweep script is
listed, and the probe reference value matches the reported runs.
This commit is contained in:
Ki-Ho Lee
2026-08-26 22:19:24 +09:00
parent 248e637f55
commit 5dff0a76a0
+52 -29
View File
@@ -1,27 +1,30 @@
# Semantic Multiplexing Gain in Wireless Systems via Expanded Embeddings: A BERT Case Study # Semantic Multiplexing Gain in Wireless Systems via Expanded Embeddings: A BERT Case Study
Code, stored results, and supplementary material for the IEEE Code, stored results, and supplementary material for the IEEE
Communications Letters submission by Ki-Ho Lee, Hyun-Ho Choi, and Communications Letters submission by Ki-Ho Lee, Hyun-Ho Choi, and
Jung-Ryun Lee. Jung-Ryun Lee.
Multiple users share one expanded embedding block of dimension Multiple users share one expanded embedding block of dimension
`d_s = K * d_b`: each user's frozen BERT sentence embedding is projected `d_s = K * d_b`. Each user's frozen BERT sentence embedding is projected
into the shared space, superimposed through learnable masks, and into the shared space, superimposed through learnable masks, and
demultiplexed by user-wise attention. All reported transceivers are demultiplexed by user-wise attention. Every transceiver reported in the
trained with SNR-aware MAML; the training without MAML of the earlier letter, including the conventional orthogonal scheme, is trained with
JSAC paper is included as a prior-art reference. SNR-aware MAML; the same architecture trained without MAML, as in the
earlier JSAC paper, is included as a prior-art reference and is labelled
"Training w/o MAML [5]" in the figures.
## Files ## Files
| File | Purpose | | File | Purpose |
|---|---| |---|---|
| `bert_semcom.py` | Shared library: BERT extractor, transceiver model, channel, MAML helpers | | `bert_semcom.py` | Shared library: BERT extractor, transceiver model, channel, MAML helpers |
| `cl_experiments.py` | Held-out split, joint-trained configurations, ToDMA token-domain benchmark, linear probe, latency | | `cl_experiments.py` | Held-out split, configurations trained without MAML, token-domain (ToDMA) comparison, linear probe, latency |
| `cl_maml_all.py` | SNR-aware MAML training for every reported configuration (including the conventional orthogonal scheme) | | `cl_maml_all.py` | SNR-aware MAML training for every reported configuration, including the conventional orthogonal scheme |
| `cl_maml_extra.py` | MAML K sweep (K = 1, 2, 8) and DistilBERT replication | | `cl_maml_extra.py` | MAML expansion-factor sweep (K = 1, 2, 8) and DistilBERT replication |
| `cl_todma_load.py` | Evaluates the token-domain scheme at U = 1, 2, 3, 5, 6 for the load sweep of Fig. 3 |
| `replot_cl.py` | Regenerates Figs. 2 and 3 of the letter from the stored JSON results | | `replot_cl.py` | Regenerates Figs. 2 and 3 of the letter from the stored JSON results |
| `probe_vs_cosine.py` | Supplementary probe-accuracy-versus-cosine-similarity analysis | | `probe_vs_cosine.py` | Supplementary probe-accuracy-versus-cosine-similarity analysis |
| `fig_cl/*.json`, `fig_cl/*.csv` | Stored raw results behind every figure and quoted number | | `fig_cl/*.json`, `fig_cl/*.csv` | Stored raw results behind every figure and every number quoted in the letter |
## Reproducing ## Reproducing
@@ -30,52 +33,72 @@ Requirements: Python 3.10+, PyTorch (CUDA), `transformers`, `datasets`,
(`fancyzhx/ag_news` fallback included). (`fancyzhx/ag_news` fallback included).
```bash ```bash
python cl_experiments.py --save-dir fig_cl # joint runs + ToDMA benchmark (~3 h on a laptop GPU) python cl_experiments.py --save-dir fig_cl # runs without MAML + token-domain comparison (~3 h on a laptop GPU)
python cl_maml_all.py --save-dir fig_cl # MAML runs (~9 h) python cl_maml_all.py --save-dir fig_cl # MAML runs (~9 h)
python cl_maml_extra.py --save-dir fig_cl # MAML K sweep + DistilBERT (~6 h) python cl_maml_extra.py --save-dir fig_cl # MAML K sweep + DistilBERT (~6 h)
python replot_cl.py # Figs. 2 and 3 from stored results python cl_todma_load.py --save-dir fig_cl # token-domain load sweep (evaluation only)
python replot_cl.py # Figs. 2 and 3 from the stored results
python probe_vs_cosine.py # supplementary analysis below python probe_vs_cosine.py # supplementary analysis below
``` ```
All experiments fix their random seeds (training seed 42, evaluation All experiments fix their random seeds (training seed 42, evaluation
seed 123, ToDMA seed 7) and evaluate on a held-out test split of 2,000 seed 123, token-domain seed 7) and evaluate on a held-out test split of
AG News sentences disjoint from the 8,000-sentence training pool. 2,000 AG News sentences disjoint from the 8,000-sentence training pool.
`replot_cl.py` and `probe_vs_cosine.py` read only the stored results, The centering mean, the transceiver parameters, and the linear probe are
so every figure is regenerable without rerunning the experiments. fitted on the training pool only. `replot_cl.py` and
`probe_vs_cosine.py` read only the stored results, so every figure is
regenerable without rerunning the experiments.
## Figures of the letter ## Figures of the letter
**Fig. 2 - per-user cosine similarity vs. SNR** (proposed scheme for **Fig. 2 - per-user cosine similarity vs. SNR.** The proposed scheme for
U = 1..4 at K = 4, the conventional orthogonal scheme, the U = 1..4 at K = 4, the conventional orthogonal scheme, the two
matched-budget schemes, and the joint training of the earlier JSAC matched-budget comparison schemes (random-projection mask and the
paper, all on the held-out test set): token-domain scheme at two slot/codeword splits), and the same
architecture trained without MAML:
![Fig. 2](fig_cl/cl_fig_mux.png) ![Fig. 2](fig_cl/cl_fig_mux.png)
**Fig. 3 - aggregate fidelity across load** (SNR-aware MAML vs. joint **Fig. 3 - aggregate fidelity across load at 10 dB.** Three bars per
training at 20 dB, with the fully loaded orthogonal reference): load point (proposed, training without MAML, and the token-domain
scheme at 24x128), the conventional single-user block on the left, and
the fully loaded orthogonal aggregate as the dash-dotted reference:
![Fig. 3](fig_cl/cl_fig_agg.png) ![Fig. 3](fig_cl/cl_fig_agg.png)
At full load the shared block stays within 1% of the fully loaded
orthogonal aggregate over the same 3072 channel uses (3.59 against
3.62), and under overload it reaches 1.45 times that aggregate at
U = 6.
## Supplementary: probe accuracy vs. cosine similarity ## Supplementary: probe accuracy vs. cosine similarity
The letter measures semantic fidelity by the cosine similarity of the The letter measures semantic fidelity by the cosine similarity of the
recovered embeddings and corroborates it with a downstream perception recovered embeddings and corroborates it with a downstream perception
metric: the AG News topic accuracy of a linear probe trained on clean metric, namely the AG News topic accuracy of a linear probe fitted on
training-pool embeddings and applied to the recovered test embeddings the training pool and applied to the recovered test embeddings. The
(clean reference about 0.855, sampling error about +/-0.01). noiseless reference is about 0.854 and the sampling error is about
+/-0.01.
Across 7 schemes x 7 SNRs (49 operating points), probe accuracy tracks Across 7 schemes x 7 SNRs (49 operating points), probe accuracy tracks
cosine similarity with a Pearson correlation of **r = 0.903**: cosine similarity with a Pearson correlation of **r = 0.903**:
![Probe accuracy vs. cosine similarity](fig_cl/probe_vs_cosine.png) ![Probe accuracy vs. cosine similarity](fig_cl/probe_vs_cosine.png)
Two readings follow. First, the low-SNR advantage of the analog Two readings follow. First, the low-SNR advantage of the embedding
embedding schemes over the token-domain scheme appears in both metrics schemes over the token-domain scheme appears in both metrics, for
(for example 0.848 vs. 0.772 in CosSim and 0.805 vs. 0.762 in accuracy example 0.848 against 0.772 in cosine similarity and 0.805 against
at 5 dB). Second, schemes within about 0.01 of each other in CosSim 0.762 in accuracy at 5 dB. Second, schemes within about 0.01 of each
differ in accuracy only on the order of the sampling error, so the other in cosine similarity differ in accuracy by at most 0.023, which
is within about two standard errors of the sampling noise, so the
cosine metric used throughout the letter is consistent with downstream cosine metric used throughout the letter is consistent with downstream
perception on this task. perception on this task.
## Citation
The letter is under review. Until it appears, please cite this
repository together with the earlier shared-embedding paper:
> K.-H. Lee, H.-H. Choi, and J.-R. Lee, "Transformer-based shared
> embedding for multiple access in semantic communications," *IEEE J.
> Sel. Areas Commun.*, vol. 44, pp. 2622-2637, 2026.