kaz29Вестник Казахстанско-Британского технического университетаHerald of the Kazakh-British Technical University1998-66882959-8109Казахстанско-Британский Технический Университет10.55452/1998-6688-2025-22-2-141-154kaz29-1995Research ArticleКОМПЬЮТЕРНЫЕ НАУКИCOMPUTER SCIENCEАДАПТАЦИЯ СТИЛЯ СОЗДАНИЯ ТЕКСТА К КОНКРЕТНОЙ АУДИТОРИИ ИЛИ СОДЕРЖАНИЮADAPTATION OF TEXT GENERATION STYLE TO A SPECIFIC AUDIENCE OR CONTENThttps://orcid.org/0009-0003-5568-6142ЖанбырбайЖ.ZhangbyrbayZh.

магистрант 

г. Алматы 

 Master’s student 

 Almaty 

z_zhangbyrbay@kbtu.kzhttps://orcid.org/0000-0002-3221-9352AхметовИ.AkhmetovI.

PhD, профессор 

г. Алматы 

 PhD, professor 

 Almaty 

i.akhmetov@ipic.kzhttps://orcid.org/0000-0002-8685-9355ПакА.PakA.

 PhD, профессор 

 г. Алматы

 PhD, professor 

 Almaty 

a.pak@kbtu.kzhttps://orcid.org/0000-0003-0422-7432ДжаксылыковаА.JaxylykovaA.

 докторант 

г. Алматы

 PhD student 

 Almaty 

a.jaxylykova@kbtu.kzhttps://orcid.org/0000-0002-9032-9285КомадаП.KomadaP.

PhD, профессор 

г. Люблин

 PhD, professor 

 Lublin 

p.komada@pollub.plКазахстанско-Британский технический университетКазахстанKazakh-British Technical UniversityKazakhstanИнститут информационных и вычислительных технологийКазахстанInstitute of Information and Computational TechnologiesKazakhstanЛюблинский технологический университетПольшаLublin University of TechnologyPoland202506072025222141154Copyright © Жанбырбай Ж., Aхметов И., Пак А., Джаксылыкова А., Комада П., 20252025Жанбырбай Ж., Aхметов И., Пак А., Джаксылыкова А., Комада П.Zhangbyrbay Z., Akhmetov I., Pak A., Jaxylykova A., Komada P.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.kbtu.edu.kz/jour/article/view/1995

Адаптация стиля генерации текста к конкретной аудитории или содержанию может быть достигнута без дорогостоящей тонкой настройки. Мы отказываемся от модельных весов и вместо этого (i) перебираем восемь гиперпараметров декодера с помощью байесовской оптимизации и (ii) добавляем однострочную стилевую подсказку, которая изменяет удобочитаемость. Эксперименты на пяти математических бенчмарках (AQUA-RAT, MathQA, GSM8K, MAWPS, SVAMP) с тремя контрольными точками с параметрами 8-14 B (LLaMA-3.1-8B, DeepSeek-Qwen-8B/14B) показали, что 50-пробный поиск Optuna повышает точность точного соответствия на 36 процентных пунктов и закрывает 5–10 пунктов разрыва с базовыми точками с точной настройкой 30–70 B. Те же настройки переносятся между задачами с потерей менее двух пунктов. Добавление заголовка, ориентированного на детей, оставляет точность практически неизменной, вдвое снижая уровень оценки по Флешу-Кинкейду и сокращая трассы рассуждений. Все эксперименты укладываются в несколько GPU-часов на одном A100, что делает метод практичным для развертывания в условиях ограниченных ресурсов. Исследование демонстрирует, что тщательный контроль декодера в сочетании с микропрограммами обеспечивает численную корректность и приемлемое для аудитории изложение без дополнительного времени на обучение или настройку.

Adaptation of text generation style to specific audiences or content can be achieved without costly fine-tuning. We freeze model weights and instead (i) search eight decoder hyperparameters with Bayesian optimization and (ii) prepend a one-line style cue that modulates readability. Experiments on five mathematical question-answering benchmarks (AQUA-RAT, MathQA, GSM8K, MAWPS, SVAMP) with three 8–14 B-parameter checkpoints (LLaMA-3.1-8B, DeepSeek-Qwen-8B/14B) show that 50-trial Optuna searches raise exact-match accuracy by up to 36 percentage points and close 5–10 points of the gap to 30–70 B fine-tuned baselines. The same settings transfer across tasks with under 2-point loss. Adding the child-friendly header leaves accuracy virtually unchanged while halving the Flesch–Kincaid grade level and shortening reasoning traces. All experiments fit within a few GPU-hours on a single A100, making the method practical for resource-constrained deployments. The study demonstrates that careful decoder control combined with micro-prompts delivers numerical correctness and audience-appropriate exposition without additional training or tuning time.

оптимизация декодераадаптация стилячитабельностьбольшие языковые моделиматематические ответы на вопросыбайесовский поиск гиперпараметровоценка Flesch-Kincaid.decoder optimizationstyle adaptationreadabilitylarge language modelsmathematical question answeringBayesian hyper-parameter searchFlesch–Kincaid scoreThis research was funded by the Science Committee of the Ministry of Science and Higher Education of the Republic of Kazakhstan (Grant No. AP23489782)This research was funded by the Science Committee of the Ministry of Science and Higher Education of the Republic of Kazakhstan (Grant No. AP23489782)ReferencesBrown T. et al. Language models are few-shot learners // Advances in neural information processing systems. – 2020. – Vol. 33. – P. 1877–1901.Brown T. et al. Language models are few-shot learners // Advances in neural information processing systems. – 2020. – Vol. 33. – P. 1877–1901.Wei J. et al. Chain-of-thought prompting elicits reasoning in large language models // Advances in neural information processing systems. – 2022. – Vol. 35. – P. 24824–24837.Wei J. et al. Chain-of-thought prompting elicits reasoning in large language models // Advances in neural information processing systems. – 2022. – Vol. 35. – P. 24824–24837.Kojima T. et al. Large language models are zero-shot reasoners // Advances in neural information processing systems. – 2022. – Vol. 35. – P. 22199–22213.Kojima T. et al. Large language models are zero-shot reasoners // Advances in neural information processing systems. – 2022. – Vol. 35. – P. 22199–22213.Touvron H. et al. Llama: Open and efficient foundation language models //arXiv preprint arXiv:2302.13971. – 2023.Touvron H. et al. Llama: Open and efficient foundation language models //arXiv preprint arXiv:2302.13971. – 2023.Holtzman A. et al. The curious case of neural text degeneration // arXiv preprint arXiv:1904.09751. – 2019.Holtzman A. et al. The curious case of neural text degeneration // arXiv preprint arXiv:1904.09751. – 2019.Wu Y. et al. Google’s neural machine translation system: Bridging the gap between human and machine translation // arXiv preprint arXiv:1609.08144. – 2016.Wu Y. et al. Google’s neural machine translation system: Bridging the gap between human and machine translation // arXiv preprint arXiv:1609.08144. – 2016.Ippolito D. et al. Comparison of diverse decoding methods from conditional language models // arXiv preprint arXiv:1906.06362. – 2019.Ippolito D. et al. Comparison of diverse decoding methods from conditional language models // arXiv preprint arXiv:1906.06362. – 2019.Bergstra J., Bengio Y. Random search for hyper-parameter optimization // The journal of machine learning research. – 2012. – Vol. 13. – No. 1. – P. 281–305.Bergstra J., Bengio Y. Random search for hyper-parameter optimization // The journal of machine learning research. – 2012. – Vol. 13. – No. 1. – P. 281–305.Snoek J., Larochelle H., Adams R.P. Practical bayesian optimization of machine learning algorithms // Advances in neural information processing systems. – 2012. – Vol. 25.Snoek J., Larochelle H., Adams R.P. Practical bayesian optimization of machine learning algorithms // Advances in neural information processing systems. – 2012. – Vol. 25.Hutter F., Hoos H. H., Leyton-Brown K. Sequential model-based optimization for general algorithm configuration // Learning and intelligent optimization: 5th international conference, LION 5, Rome, Italy, January 17–21, 2011. selected papers 5. – Springer Berlin Heidelberg, 2011. – P. 507–523.Hutter F., Hoos H. H., Leyton-Brown K. Sequential model-based optimization for general algorithm configuration // Learning and intelligent optimization: 5th international conference, LION 5, Rome, Italy, January 17–21, 2011. selected papers 5. – Springer Berlin Heidelberg, 2011. – P. 507–523.Akiba T. et al. Optuna: A next-generation hyperparameter optimization framework // Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. – 2019. – P. 2623–2631.Akiba T. et al. Optuna: A next-generation hyperparameter optimization framework // Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. – 2019. – P. 2623–2631.Falkner S., Klein A., Hutter F. BOHB: Robust and efficient hyperparameter optimization at scale // International conference on machine learning. – PMLR, 2018. – P. 1437–1446.Falkner S., Klein A., Hutter F. BOHB: Robust and efficient hyperparameter optimization at scale // International conference on machine learning. – PMLR, 2018. – P. 1437–1446.Wang X. et al. H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models //The Eleventh International Conference on Learning Representations. – 2023. – Vol. 1.Wang X. et al. H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models //The Eleventh International Conference on Learning Representations. – 2023. – Vol. 1.Amini A. et al. Mathqa: Towards interpretable math word problem solving with operation-based formalisms // arXiv preprint arXiv:1905.13319. – 2019.Amini A. et al. Mathqa: Towards interpretable math word problem solving with operation-based formalisms // arXiv preprint arXiv:1905.13319. – 2019.Patel A., Bhattamishra S., Goyal N. Are NLP models really able to solve simple math word problems? //arXiv preprint arXiv:2103.07191. – 2021.Patel A., Bhattamishra S., Goyal N. Are NLP models really able to solve simple math word problems? //arXiv preprint arXiv:2103.07191. – 2021.Ling W. et al. Program induction by rationale generation: Learning to solve and explain algebraic word problems //arXiv preprint arXiv:1705.04146. – 2017.Ling W. et al. Program induction by rationale generation: Learning to solve and explain algebraic word problems //arXiv preprint arXiv:1705.04146. – 2017.Cobbe K. et al. Training verifiers to solve math word problems //arXiv preprint arXiv:2110.14168. – 2021.Cobbe K. et al. Training verifiers to solve math word problems //arXiv preprint arXiv:2110.14168. – 2021.Koncel-Kedziorski R. et al. MAWPS: A math word problem repository //Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: human language technologies. – 2016. – P. 1152–1157.Koncel-Kedziorski R. et al. MAWPS: A math word problem repository //Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: human language technologies. – 2016. – P. 1152–1157.Gao L. et al. Pal: Program-aided language models // International Conference on Machine Learning. – PMLR, 2023. – P. 10764–10799.Gao L. et al. Pal: Program-aided language models // International Conference on Machine Learning. – PMLR, 2023. – P. 10764–10799.Lewkowycz A. et al. Solving quantitative reasoning problems with language models //Advances in Neural Information Processing Systems. – 2022. – Vol. 35. – P. 3843–3857.Lewkowycz A. et al. Solving quantitative reasoning problems with language models //Advances in Neural Information Processing Systems. – 2022. – Vol. 35. – P. 3843–3857.Hendrycks D. et al. Measuring mathematical problem solving with the math dataset // arXiv preprint arXiv:2103.03874. – 2021.Hendrycks D. et al. Measuring mathematical problem solving with the math dataset // arXiv preprint arXiv:2103.03874. – 2021.Feurer M., Hutter F. Hyperparameter optimization. – Springer International Publishing, 2019. – P. 3–33.Feurer M., Hutter F. Hyperparameter optimization. – Springer International Publishing, 2019. – P. 3–33.Bergstra J., Yamins D., Cox D. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures // International conference on machine learning. – PMLR, 2013. – P. 115–123.Bergstra J., Yamins D., Cox D. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures // International conference on machine learning. – PMLR, 2013. – P. 115–123.Li L. et al. Hyperband: A novel bandit-based approach to hyperparameter optimization // Journal of Machine Learning Research. – 2018. – Vol. 18. – No. 185. – P. 1–52.Li L. et al. Hyperband: A novel bandit-based approach to hyperparameter optimization // Journal of Machine Learning Research. – 2018. – Vol. 18. – No. 185. – P. 1–52.Fan A., Lewis M., Dauphin Y. Hierarchical neural story generation // arXiv preprint arXiv:1805.04833. – 2018.Fan A., Lewis M., Dauphin Y. Hierarchical neural story generation // arXiv preprint arXiv:1805.04833. – 2018.Keskar N. S. et al. Ctrl: A conditional transformer language model for controllable generation //arXiv preprint arXiv:1909.05858. – 2019.Keskar N. S. et al. Ctrl: A conditional transformer language model for controllable generation //arXiv preprint arXiv:1909.05858. – 2019.Pillutla K. et al. Mauve: Measuring the gap between neural text and human text using divergence frontiers // Advances in Neural Information Processing Systems. – 2021. – Vol. 34. – P. 4816–4828.Pillutla K. et al. Mauve: Measuring the gap between neural text and human text using divergence frontiers // Advances in Neural Information Processing Systems. – 2021. – Vol. 34. – P. 4816–4828.Shi C. et al. A thorough examination of decoding methods in the era of llms // arXiv preprint arXiv:2402.06925. – 2024.Shi C. et al. A thorough examination of decoding methods in the era of llms // arXiv preprint arXiv:2402.06925. – 2024.

The authors declare that there are no conflicts of interest present.