C++ Read File Line by Line With Delimiter
Solarian Developer
My programming ramblings
C Programming - read a file line past line with fgets and getline, implement a portable getline version
Posted on Apr 3, 2019 by Paul
In this article, I will show y'all how to read a text file line past line in C using the standard C function fgets and the POSIX getline function. At the end of the commodity, I will write a portable implementation of the getline part that can be used with any standard C compiler.
Reading a file line past line is a lilliputian problem in many programming languages, simply not in C. The standard mode of reading a line of text in C is to use the fgets function, which is fine if you know in advance how long a line of text could exist.
You tin can detect all the lawmaking examples and the input file at the GitHub repo for this article.
Let's showtime with a unproblematic example of using fgets to read chunks from a text file. :
1 #include <stdio.h> two #include <stdlib.h> 3 four int main ( void ) { five FILE * fp = fopen ( "lorem.txt" , "r" ); 6 if ( fp == NULL ) { seven perror ( "Unable to open up file!" ); 8 exit ( 1 ); 9 } ten eleven char chunk [ 128 ]; 12 thirteen while ( fgets ( chunk , sizeof ( chunk ), fp ) != Naught ) { fourteen fputs ( chunk , stdout ); 15 fputs ( "|* \due north " , stdout ); // marker string used to evidence where the content of the chunk assortment has ended 16 } 17 18 fclose ( fp ); nineteen } For testing the code I've used a simple dummy file, lorem.txt. This is a slice from the output of the above program on my machine:
1 ~ $ clang -std=c17 -Wall -Wextra -pedantic t0.c -o t0 2 ~ $ ./t0 iii Lorem ipsum dolor sit amet, consectetur adipiscing elit. 4 |* 5 Fusce dignissim facilisis ligula consectetur hendrerit. Vestibulum porttitor aliquam luctus. Nam pharetra lorem vel ornare cond|* half dozen imentum. 7 |* viii Praesent et nunc at libero vulputate convallis. Cras egestas nunc vitae eros vehicula hendrerit. Pellentesque in est et sapien |* 9 dignissim molestie. 10 |* The lawmaking prints the content of the chunk array, as filled after every call to fgets, and a mark string.
If you lookout man carefully, by scrolling the above text snippet to the correct, yous can meet that the output was truncated to 127 characters per line of text. This was expected because our code tin can shop an unabridged line from the original text file only if the line tin can fit within our chunk assortment.
What if you demand to accept the entire line of text bachelor for further processing and not a slice of line ? A possible solution is to copy or concatenate chunks of text in a separate line buffer until we notice the end of line character.
Let'south start past creating a line buffer that will store the chunks of text, initially this will take the same length as the chunk array:
1 #include <stdio.h> 2 #include <stdlib.h> iii #include <string.h> 4 5 int primary ( void ) { 6 FILE * fp = fopen ( "lorem.txt" , "r" ); seven // ... viii 9 char chunk [ 128 ]; 10 eleven // Store the chunks of text into a line buffer 12 size_t len = sizeof ( chunk ); 13 char * line = malloc ( len ); xiv if ( line == Goose egg ) { 15 perror ( "Unable to allocate memory for the line buffer." ); xvi leave ( ane ); 17 } 18 19 // "Empty" the cord 20 line [ 0 ] = '\0' ; 21 22 // ... 23 24 } Next, we are going to append the content of the chunk array to the cease of the line string, until we find the end of line graphic symbol. If necessary, we'll resize the line buffer:
1 #include <stdio.h> 2 #include <stdlib.h> 3 #include <cord.h> iv v int main ( void ) { half-dozen // ... 7 8 // "Empty" the string 9 line [ 0 ] = '\0' ; 10 11 while ( fgets ( chunk , sizeof ( chunk ), fp ) != Goose egg ) { 12 // Resize the line buffer if necessary 13 size_t len_used = strlen ( line ); 14 size_t chunk_used = strlen ( chunk ); 15 xvi if ( len - len_used < chunk_used ) { 17 len *= 2 ; xviii if (( line = realloc ( line , len )) == Zippo ) { 19 perror ( "Unable to reallocate memory for the line buffer." ); 20 free ( line ); 21 exit ( one ); 22 } 23 } 24 25 // Re-create the chunk to the end of the line buffer 26 strncpy ( line + len_used , chunk , len - len_used ); 27 len_used += chunk_used ; 28 29 // Check if line contains '\northward', if yes process the line of text xxx if ( line [ len_used - one ] == '\n' ) { 31 fputs ( line , stdout ); 32 fputs ( "|* \n " , stdout ); 33 // "Empty" the line buffer 34 line [ 0 ] = '\0' ; 35 } 36 } 37 38 fclose ( fp ); 39 free ( line ); 40 41 printf ( " \n\n Max line size: %zd \due north " , len ); 42 } Delight note, that in the above lawmaking, every time the line buffer needs to be resized its chapters is doubled.
This is the result of running the to a higher place code on my machine. For brevity, I kept only the offset lines of output:
i ~ $ clang -std=c17 -Wall -Wextra -pedantic t1.c -o t1 two ~ $ ./t1 three Lorem ipsum dolor sit down amet, consectetur adipiscing elit. iv |* 5 Fusce dignissim facilisis ligula consectetur hendrerit. Vestibulum porttitor aliquam luctus. Nam pharetra lorem vel ornare condimentum. vi |* 7 Praesent et nunc at libero vulputate convallis. Cras egestas nunc vitae eros vehicula hendrerit. Pellentesque in est et sapien dignissim molestie. 8 |* 9 Aliquam erat volutpat. Mauris dignissim augue ac purus placerat scelerisque. Donec eleifend ut nibh eu elementum. 10 |* You tin run across that, this time, we tin can impress full lines of text and not stock-still length chunks like in the initial approach.
Let's modify the above code in order to print the line length instead of the bodily text:
one // ... 2 iii int main ( void ) { 4 // ... five half dozen while ( fgets ( chunk , sizeof ( clamper ), fp ) != NULL ) { 7 8 // ... 9 x // Cheque if line contains '\n', if yes process the line of text xi if ( line [ len_used - 1 ] == '\due north' ) { 12 printf ( "line length: %zd \n " , len_used ); 13 // "Empty" the line buffer xiv line [ 0 ] = '\0' ; 15 } sixteen } 17 18 fclose ( fp ); 19 free ( line ); 20 21 printf ( " \n\n Max line size: %zd \n " , len ); 22 } This is the result of running the modified code on my machine:
1 ~ $ clang -std=c17 -Wall -Wextra -pedantic t1.c -o t1 2 ~ $ ./t1 3 line length: 57 iv line length: 136 five line length: 147 6 line length: 114 7 line length: 112 8 line length: 95 nine line length: 62 ten line length: 1 eleven line length: 428 12 line length: 1 13 line length: 460 14 line length: 1 fifteen line length: 834 sixteen line length: ane 17 line length: 821 xviii nineteen twenty Max line size: 1024 In the next instance, I volition show y'all how to employ the getline function available on POSIX systems like Linux, Unix and macOS. Microsoft Visual Studio doesn't take an equivalent function, and so you won't be able to easily exam this example on a Windows system. However, you should be able to exam it if you lot are using Cygwin or Windows Subsystem for Linux.
1 #include <stdio.h> 2 #include <stdlib.h> iii #include <string.h> four five int master ( void ) { vi FILE * fp = fopen ( "lorem.txt" , "r" ); 7 if ( fp == Zero ) { 8 perror ( "Unable to open file!" ); ix exit ( 1 ); x } xi 12 // Read lines using POSIX function getline thirteen // This lawmaking won't work on Windows 14 char * line = NULL ; 15 size_t len = 0 ; 16 17 while ( getline ( & line , & len , fp ) != - 1 ) { xviii printf ( "line length: %zd \north " , strlen ( line )); 19 } 20 21 printf ( " \n\n Max line size: %zd \n " , len ); 22 23 fclose ( fp ); 24 free ( line ); // getline will resize the input buffer every bit necessary 25 // the user needs to costless the retention when non needed! 26 } Delight note, how simple is to use POSIX's getline versus manually buffering chunks of line like in my previous instance. It is unfortunate that the standard C library doesn't include an equivalent role.
When you use getline, don't forget to costless the line buffer when yous don't need information technology anymore. Besides, calling getline more than once will overwrite the line buffer, make a copy of the line content if y'all need to go on it for further processing.
This is the result of running the above getline instance on a Linux auto:
ane ~ $ clang -std=gnu17 -Wall -Wextra -pedantic t2.c -o t2 2 ~ $ ./t2 3 line length: 57 iv line length: 136 five line length: 147 6 line length: 114 7 line length: 112 8 line length: 95 9 line length: 62 10 line length: ane 11 line length: 428 12 line length: ane 13 line length: 460 14 line length: 1 15 line length: 834 16 line length: one 17 line length: 821 xviii 19 20 Max line size: 960 It is interesting to notation, that for this particular case the getline function on Linux resizes the line buffer to a max of 960 bytes. If you run the same code on macOS the line buffer is resized to 1024 bytes. This is due to the unlike ways in which getline is implemented on different Unix like systems.
As mentioned before, getline is non present in the C standard library. It could exist an interesting exercise to implement a portable version of this part. The idea here is non to implement the about performant version of getline, simply rather to implement a elementary replacement for not POSIX systems.
Nosotros are going to have the higher up example and replace the POSIX's getline version with our ain implementation, say my_getline. Obviously, if you are on a POSIX system, you lot should utilise the version provided by the operating organization, which was tested by countless users and tuned for optimal performance.
The POSIX getline office has this signature:
1 ssize_t getline ( char ** restrict lineptr , size_t * restrict north , FILE * restrict stream ); Since ssize_t is as well a POSIX defined type, usually a 64 bits signed integer, this is how we are going to declare our version:
ane int64_t my_getline ( char ** restrict line , size_t * restrict len , FILE * restrict fp ); In principle we are going to implement the function using the same arroyo as in one of the in a higher place examples, where I've defined a line buffer and kept copying chunks of text in the buffer until we found the end of line grapheme:
1 // This will but have effect on Windows with MSVC 2 #ifdef _MSC_VER 3 #define _CRT_SECURE_NO_WARNINGS 1 4 #define restrict __restrict 5
0 Response to "C++ Read File Line by Line With Delimiter"
Post a Comment