Reset OMC Log Analytics file collections

Developing a series of log analytics log parsers and log sources is trickey at the best of times.  Rather than using the on-demand upload I wanted the Oracle Management Cloud agent to pick up the files.

As is almost always the case I had to delete the data and try again, several times.  If you don't delete the data first you will end up with duplicate log records.

Deleting the data was easy.

First of all remove the association to the entity that the log source is tied to, then goto the Log Admin section and define a query to delete all the previously uploaded data, in my case that was:-

'Log Source'='OIC Connectivity Agent Log Source'

Unfortunately log analytics has a bunch of internal state files that keep a record of what has already been processed.

These files are contained in $AGENT_INST/sysman/emd/state/loganalytics/os_file and contain data similar to:-

version=154

dirName=/home/oracle/OICS_CONNECTIVITYAGENT02/agenthome/logs

isLargeDir=false

forwardTime=0

forwardFileName=null

backwardTime=0

backwardFileName=null

lastReadCycleTime=1631087347454

################################

agent-diagnostic7.log;80467459;1629238676000;10485951;10485951;ISO-8859-1;UTC;2021-08-14T14:12:42.991Z;0

agent-diagnostic5.log;80467469;1629814398000;10485780;10485780;ISO-8859-1;UTC;2021-08-21T06:15:54.435Z;0

agent-diagnostic6.log;80467468;1629526554000;10486076;10486076;ISO-8859-1;UTC;2021-08-17T22:17:56.123Z;0

agent-diagnostic8.log;80467467;1628950362000;10485898;10485898;ISO-8859-1;UTC;2021-08-11T05:50:17.475Z;0

agent-diagnostic3.log;80467462;1630389762000;10485783;10485783;ISO-8859-1;UTC;2021-08-27T22:07:46.938Z;0

agent-diagnostic4.log;80467470;1630102066000;10485844;10485844;ISO-8859-1;UTC;2021-08-24T14:13:18.831Z;0

agent-diagnostic9.log;80467466;1628661017000;10485860;10485860;ISO-8859-1;UTC;2021-08-09T04:44:02.439Z;0

agent-diagnostic0.log;80467465;1631087314000;5765858;5765858;ISO-8859-1;UTC;2021-09-06T21:52:22.778Z;0

agent-diagnostic1.log;80467464;1630965142000;10485953;10485953;ISO-8859-1;UTC;2021-09-03T13:57:31.951Z;0

agent-diagnostic2.log;80467463;1630677451000;10485791;10485791;ISO-8859-1;UTC;2021-08-31T06:02:42.259Z;0

agent-upgrade.log;69530634;1631011778000;552233;552233;ISO-8859-1;UTC;2021-07-31T19:49:30.218Z;0

agent-audit.log.1;69832466;1625572011000;6500;6500;ISO-8859-1;UTC;2021-01-06T14:37:06.813Z;0

agent-alert.log;69530630;1631033124000;734154;734154;ISO-8859-1;UTC;2021-08-01T17:24:50.482Z;0

agent-audit.log;69530632;1626101136000;553854;553854;ISO-8859-1;UTC;2020-08-21T11:15:12.950Z;0

Stop the OMC Agent using omcli stop agent

Delete the file associated for your log source

Start the OMC Agent using omcli start agent

Re-Associate the entity using the OMC console.