How can I iterate through each line of a file using each line to replace a string in other files, creating a...












0















I have file 1 which contains:



AAAAAA
BBBBBB


I have file 2 which contains:



11111 22222 33333


and file 3 which contains:



22222 11111 33333


I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:



File 1:



AAAAAA2222233333


File 2:



BBBBBB2222233333


File 3:



22222AAAAAA33333


File 4:



22222BBBBBB33333


I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)



FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do

cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt

done

done


Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.










share|improve this question

























  • There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

    – jonny b
    1 hour ago






  • 2





    Why do the AAAAA and BBBBB appear in the same location in each pair of files, regardless of whether the 111111 is at the beginning or the middle?

    – steeldriver
    1 hour ago








  • 1





    And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

    – terdon
    1 hour ago











  • Apologies, my mistake, i have corrected them.

    – jonny b
    58 mins ago
















0















I have file 1 which contains:



AAAAAA
BBBBBB


I have file 2 which contains:



11111 22222 33333


and file 3 which contains:



22222 11111 33333


I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:



File 1:



AAAAAA2222233333


File 2:



BBBBBB2222233333


File 3:



22222AAAAAA33333


File 4:



22222BBBBBB33333


I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)



FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do

cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt

done

done


Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.










share|improve this question

























  • There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

    – jonny b
    1 hour ago






  • 2





    Why do the AAAAA and BBBBB appear in the same location in each pair of files, regardless of whether the 111111 is at the beginning or the middle?

    – steeldriver
    1 hour ago








  • 1





    And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

    – terdon
    1 hour ago











  • Apologies, my mistake, i have corrected them.

    – jonny b
    58 mins ago














0












0








0








I have file 1 which contains:



AAAAAA
BBBBBB


I have file 2 which contains:



11111 22222 33333


and file 3 which contains:



22222 11111 33333


I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:



File 1:



AAAAAA2222233333


File 2:



BBBBBB2222233333


File 3:



22222AAAAAA33333


File 4:



22222BBBBBB33333


I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)



FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do

cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt

done

done


Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.










share|improve this question
















I have file 1 which contains:



AAAAAA
BBBBBB


I have file 2 which contains:



11111 22222 33333


and file 3 which contains:



22222 11111 33333


I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:



File 1:



AAAAAA2222233333


File 2:



BBBBBB2222233333


File 3:



22222AAAAAA33333


File 4:



22222BBBBBB33333


I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)



FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do

cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt

done

done


Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.







bash shell-script sed






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 3 mins ago









Jeff Schaller

39.5k1054126




39.5k1054126










asked 2 hours ago









jonny bjonny b

93




93













  • There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

    – jonny b
    1 hour ago






  • 2





    Why do the AAAAA and BBBBB appear in the same location in each pair of files, regardless of whether the 111111 is at the beginning or the middle?

    – steeldriver
    1 hour ago








  • 1





    And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

    – terdon
    1 hour ago











  • Apologies, my mistake, i have corrected them.

    – jonny b
    58 mins ago



















  • There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

    – jonny b
    1 hour ago






  • 2





    Why do the AAAAA and BBBBB appear in the same location in each pair of files, regardless of whether the 111111 is at the beginning or the middle?

    – steeldriver
    1 hour ago








  • 1





    And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

    – terdon
    1 hour ago











  • Apologies, my mistake, i have corrected them.

    – jonny b
    58 mins ago

















There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

– jonny b
1 hour ago





There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

– jonny b
1 hour ago




2




2





Why do the AAAAA and BBBBB appear in the same location in each pair of files, regardless of whether the 111111 is at the beginning or the middle?

– steeldriver
1 hour ago







Why do the AAAAA and BBBBB appear in the same location in each pair of files, regardless of whether the 111111 is at the beginning or the middle?

– steeldriver
1 hour ago






1




1





And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

– terdon
1 hour ago





And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

– terdon
1 hour ago













Apologies, my mistake, i have corrected them.

– jonny b
58 mins ago





Apologies, my mistake, i have corrected them.

– jonny b
58 mins ago










2 Answers
2






active

oldest

votes


















0














You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



!/bin/bash

i=1
FILES=./other_file*
while read r; do
for f in $FILES; do
sed "s/11111/$r/g" > newfiles/file$i.txt < $f
i=$(( i + 1 ))
done
done <file1


Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.






share|improve this answer































    0














    How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



    awk -v OFS= '
    NR==FNR {t[NR]=$0; next}
    {s = $0}
    {
    for(i in t) {
    $0 = s;
    sub(/11111/,t[i]);
    $1=$1;
    print > "newfile" ++n
    }
    }' file1 file2 file3


    (the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



    $ head newfile*
    ==> newfile1 <==
    AAAAAA2222233333

    ==> newfile2 <==
    BBBBBB2222233333

    ==> newfile3 <==
    22222AAAAAA33333

    ==> newfile4 <==
    22222BBBBBB33333





    share|improve this answer























      Your Answer








      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "106"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495539%2fhow-can-i-iterate-through-each-line-of-a-file-using-each-line-to-replace-a-strin%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      2 Answers
      2






      active

      oldest

      votes








      2 Answers
      2






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      0














      You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



      !/bin/bash

      i=1
      FILES=./other_file*
      while read r; do
      for f in $FILES; do
      sed "s/11111/$r/g" > newfiles/file$i.txt < $f
      i=$(( i + 1 ))
      done
      done <file1


      Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.






      share|improve this answer




























        0














        You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



        !/bin/bash

        i=1
        FILES=./other_file*
        while read r; do
        for f in $FILES; do
        sed "s/11111/$r/g" > newfiles/file$i.txt < $f
        i=$(( i + 1 ))
        done
        done <file1


        Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.






        share|improve this answer


























          0












          0








          0







          You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



          !/bin/bash

          i=1
          FILES=./other_file*
          while read r; do
          for f in $FILES; do
          sed "s/11111/$r/g" > newfiles/file$i.txt < $f
          i=$(( i + 1 ))
          done
          done <file1


          Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.






          share|improve this answer













          You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



          !/bin/bash

          i=1
          FILES=./other_file*
          while read r; do
          for f in $FILES; do
          sed "s/11111/$r/g" > newfiles/file$i.txt < $f
          i=$(( i + 1 ))
          done
          done <file1


          Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered 34 mins ago









          user1794469user1794469

          1,4421822




          1,4421822

























              0














              How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



              awk -v OFS= '
              NR==FNR {t[NR]=$0; next}
              {s = $0}
              {
              for(i in t) {
              $0 = s;
              sub(/11111/,t[i]);
              $1=$1;
              print > "newfile" ++n
              }
              }' file1 file2 file3


              (the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



              $ head newfile*
              ==> newfile1 <==
              AAAAAA2222233333

              ==> newfile2 <==
              BBBBBB2222233333

              ==> newfile3 <==
              22222AAAAAA33333

              ==> newfile4 <==
              22222BBBBBB33333





              share|improve this answer




























                0














                How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



                awk -v OFS= '
                NR==FNR {t[NR]=$0; next}
                {s = $0}
                {
                for(i in t) {
                $0 = s;
                sub(/11111/,t[i]);
                $1=$1;
                print > "newfile" ++n
                }
                }' file1 file2 file3


                (the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



                $ head newfile*
                ==> newfile1 <==
                AAAAAA2222233333

                ==> newfile2 <==
                BBBBBB2222233333

                ==> newfile3 <==
                22222AAAAAA33333

                ==> newfile4 <==
                22222BBBBBB33333





                share|improve this answer


























                  0












                  0








                  0







                  How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



                  awk -v OFS= '
                  NR==FNR {t[NR]=$0; next}
                  {s = $0}
                  {
                  for(i in t) {
                  $0 = s;
                  sub(/11111/,t[i]);
                  $1=$1;
                  print > "newfile" ++n
                  }
                  }' file1 file2 file3


                  (the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



                  $ head newfile*
                  ==> newfile1 <==
                  AAAAAA2222233333

                  ==> newfile2 <==
                  BBBBBB2222233333

                  ==> newfile3 <==
                  22222AAAAAA33333

                  ==> newfile4 <==
                  22222BBBBBB33333





                  share|improve this answer













                  How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



                  awk -v OFS= '
                  NR==FNR {t[NR]=$0; next}
                  {s = $0}
                  {
                  for(i in t) {
                  $0 = s;
                  sub(/11111/,t[i]);
                  $1=$1;
                  print > "newfile" ++n
                  }
                  }' file1 file2 file3


                  (the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



                  $ head newfile*
                  ==> newfile1 <==
                  AAAAAA2222233333

                  ==> newfile2 <==
                  BBBBBB2222233333

                  ==> newfile3 <==
                  22222AAAAAA33333

                  ==> newfile4 <==
                  22222BBBBBB33333






                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered 29 mins ago









                  steeldriversteeldriver

                  35.6k35286




                  35.6k35286






























                      draft saved

                      draft discarded




















































                      Thanks for contributing an answer to Unix & Linux Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495539%2fhow-can-i-iterate-through-each-line-of-a-file-using-each-line-to-replace-a-strin%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      Histoire des bourses de valeurs

                      Why is there Russian traffic in my log files?

                      Rename multiple files to decrement number in file name?