json - Sed'ing fails while replacing only first occurence of string in a file with complex string

Question

I have a json file (config.json):

{
"Repo": {
        "docker-node" : {
            "ChangelogMsg": "### Changed
- Base container moved to BaseOS 10.9.0

### Security
- Includes official release of nodejs 14.15.4-1 with security fixes for [CVE-2020-8265](https://nvd.nist.gov/vuln/detail/CVE-2020-8265), [CVE-2020-1971](https://nvd.nist.gov/vuln/detail/CVE-2020-1971) and [CVE-2020-8287](https://nvd.nist.gov/vuln/detail/CVE-2020-8287)

",
            "Tag": "buster-14.15.4-2"
        }
    }
}

I am trying to update changelog file with value of property: Repo.docker-node.ChangelogMsg

I use JQuery to fetch the value of required property:

repo=docker-node
CONFIG=path/to/config.json
CHANGELOG_MSG=$(jq -r --arg repovar "$repo" '.Repo[$repovar].ChangelogMsg | strings' $CONFIG)
REL_TAG=$(jq -r --arg repovar "$repo" '.Repo[$repovar].Tag | strings' $CONFIG)
CHLOG_STR=$(printf "[$REL_TAG] - $DATE
$CHANGELOG_MSG

## ")

And, try using following approaches to update changelog file:

Sample CHANGELOG.md:

# Change Log
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/).

## [buster-14.15.4-1] - 2021-01-12
### Changed
- Updated to latest nodejs version on 14.x series i.e. 14.15.4-1. This brings in new features and functionalities along with some incompatibilities. For more info, see the [release notes](https://nodejs.org/en/blog/release/v14.0.0/)

Using sed:

# Replace "## " with changelog string for only first occurance in file CHANGELOG.md
sed "1,|## |s||${CHLOG_STR}|" /path/to/CHANGELOG.md

but receive:

sed: -e expression #1, char 3: unexpected `,'

Using awk:

awk -v var1="## " -v var2="$CHLOG_STR" '{sub(var1,var2,$0); print $0}1' /path/to/CHANGELOG.md

but above command will not write/update file. Using -i option, but is not recognized, so I use:

awk -v var1="## " -v var2="$CHLOG_STR" '{sub(var1,var2,$0); print $0}1' /path/to/CHANGELOG.md > temp.txt && mv temp.txt /path/to/CHANGELOG.md

Although it do update CHANGELOG.md, but duplicates every line (as below) and globally replaces "## " instead of just the first occurance.

# Change Log
# Change Log
All notable changes to this project will be documented in this file.
All notable changes to this project will be documented in this file.


The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/).
The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/).

Here is the expected output:

# Change Log
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/).

## [buster-14.15.4-2] - 2021-01-23
### Changed
- Base container moved to BaseOS 10.9.0

### Security
- Includes official release of nodejs 14.15.4-1 with security fixes for [CVE-2020-8265](https://nvd.nist.gov/vuln/detail/CVE-2020-8265), [CVE-2020-1971](https://nvd.nist.gov/vuln/detail/CVE-2020-1971) and [CVE-2020-8287](https://nvd.nist.gov/vuln/detail/CVE-2020-8287)

## [buster-14.15.4-1] - 2021-01-12
### Changed
- Updated to latest nodejs version on 14.x series i.e. 14.15.4-1. This brings in new features and functionalities along with some incompatibilities. For more info, see the [release notes](https://nodejs.org/en/blog/release/v14.0.0/)

Having used all the tricks and tips available online, I am looking for the pointers to resolve same. Do you have idea?

question from:https://stackoverflow.com/questions/65856426/seding-fails-while-replacing-only-first-occurence-of-string-in-a-file-with-comp

Answer 1

We cannot put newline characters in the replacement string with sed in a straightforward way. We need to escape them one by one. Otherwise sed considers the newline character as the end of command, not a part of the replacement string and causes an error. Then awk will be a better choice for the task. Please try something like :

awk -v var1="## " -v var2="$CHLOG_STR" '{ sub("^"var1, var1 var2); print }' /path/to/CHANGELOG.md

Output:

# Change Log
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/).

## [buster-14.15.4-2] - 2021-01-23
### Changed
- Base container moved to BaseOS 10.9.0

### Security
- Includes official release of nodejs 14.15.4-1 with security fixes for [CVE-2020-8265](https://nvd.nist.gov/vuln/detail/CVE-2020-8265), [CVE-2020-1971](https://nvd.nist.gov/vuln/detail/CVE-2020-1971) and [CVE-2020-8287](https://nvd.nist.gov/vuln/detail/CVE-2020-8287)

## [buster-14.15.4-1] - 2021-01-12
### Changed
- Updated to latest nodejs version on 14.x series i.e. 14.15.4-1. This brings in new features and functionalities along with some incompatibilities. For more info, see the [release notes](https://nodejs.org/en/blog/release/v14.0.0/)

The problems in your awk script are:

• You need to prepend the anchor ^ to the regex ## to make it match the start of the string. Otherwise it will match ### too.
• You are putting both print $0 and 1 which double the output.