Remove multiple occurrences of similar lines except the last

The issue with your tac approach is that it collapses all timestamp groups into one. You need to track when a timestamp block ends. I would also avoid hard-coding 2026 as you might get dates from other years.

awk '/^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}/ { last=$0; next }
     { if (last!="") { print last; last="" }; print }
     END { if (last!="") print last }' file.txt

When a timestamp line is seen, store it in last and skip printing. When a non-timestamp line is seen, if there's a stored last it means the run just ended so print it, then print the current line. The END block handles the edge case where the file ends on a timestamp.

This might work for you (GNU sed):

sed -E '/^2026/{N;/^(.{4}).*\n\1/D}' file

Set the extended regex option -E.

If the first 4 characters of a line begin 2026, append the next line.

If that line meets the criteria of a repeated one, delete the first and repeat.

N.B. The regex can be tightened up as desired e.g.

sed -E '/^....-..-.....:/{N;/^(.{13}).*\n\1/D}' file

I would harness GNU AWK for this task using 2-pass approach, let file.txt content be

important information 1
important information 2
2026-07-10T11:12:31.013620-04:00     <-- delete
2026-07-10T11:13:34.058824-04:00     <-- delete
2026-07-10T11:14:37.116946-04:00     <-- delete
2026-07-10T11:15:09.647148-04:00     <-- keep
important information 3
important information 4
important information 5
2026-07-10T11:15:12.662378-04:00     <-- keep
important information 6
important information 7
2026-07-10T11:15:18.232970-04:00     <-- keep
important information 8
2026-07-10T11:15:40.152816-04:00     <-- delete
2026-07-10T11:16:43.190958-04:00     <-- delete
2026-07-10T11:17:46.237995-04:00     <-- delete
2026-07-10T11:18:49.282324-04:00     <-- delete
2026-07-10T11:19:52.327966-04:00     <-- delete
2026-07-10T11:28:16.700459-04:00     <-- delete
2026-07-10T11:29:19.748261-04:00     <-- delete
2026-07-10T11:30:22.788192-04:00     <-- keep
important information 9
important information 10
important information 11

then

awk 'FNR==NR{arr[FNR]=/^2026/;next}!(/^2026/&&arr[FNR+1])' file.txt file.txt

gives output

important information 1
important information 2
2026-07-10T11:15:09.647148-04:00     <-- keep
important information 3
important information 4
important information 5
2026-07-10T11:15:12.662378-04:00     <-- keep
important information 6
important information 7
2026-07-10T11:15:18.232970-04:00     <-- keep
important information 8
2026-07-10T11:30:22.788192-04:00     <-- keep
important information 9
important information 10
important information 11

Explanation: during 1st pass (i.e. when number of line is equal to number of line inside file) I store information inside array arr such key is number of line and value is information if it is date line (regular expression might require adjustment if you can have other years than 2026) and instruct GNU AWK to go next line. 2nd pass is just filtering, so only line which are NOT (starting with 2026 AND following line is date line) are printed. Observer that brackets are required here as NOT has precedence over AND. DISCLAIMER: this solution assumes you have enough memory to store is-date-line information for your whole file, if this does not hold ignore this answer entirely.

(tested in GNU Awk 5.3.1)

Updated to handle comment:

In my scenario, if the last line was a timestamp, then I would not be interested in it`

The original approach (at end), assumed that the last of every run of lines matching the regex should be printed. To handle the case when such a run must be followed by a line that doesn't match the regex, we can flip the initial value of f so that it starts out false (zero):

tac file.txt | awk '/2026-/ ? !--f : (f=1)' | tac

• reverse the file
• loop over each line:
  • does line match regex?
    • yes: print line if it is first of a run after a line that didn't match the regex
      • --f decrements from 1 (0, -1, -2, ...) if the run follows a non-regex line, or from 0 (-1, -2, -3, ...) when f has never been set. its negation is true only when --f==0 which can only happen in the first of those two cases
    • no: (re)set f and print anyway
• un-reverse the result

The tac approach needs to reset f when the regex does not match.

For example:

tac file.txt | awk '!( /^2026-/ ? f++ : (f=0) )' | tac

• f is zero when program starts
• does line match regex?
  • if so, increment f and return its previous value
    • value returned will only be zero after first increment (ie. after first regex match)
  • else, reset f and return zero
    • (for portability, assignment is in parentheses to avoid ambiguous parsing)
• negate the returned value - if non-zero, print line

Or same logic but possibly easier to follow (also shorter):

tac file.txt | awk '/^2026-/ ? !f++ : !(f=0)' | tac

• does line match regex?
  • yes: print it if this is the first time f has been incremented (since a reset)
  • no: reset f and print the line