Begin log hydration

Follow these steps to begin log hydration:

1. Click the Logs tab, and then the Hydration option in the dropdown menu.
   
   

   Open

   

   Navigate to Log Hydration page

2. The Logs Hydration interface opens on the Archives tab.
   
   

   Open

   

   Logs Hydration view

   Every archive includes:

   1. First archived record (date and time): This is the hour-granularity-location of the oldest archive object. Kloudfuse uses this timestamp by default when starting the hydration job. You can override this setting and set a differnt start time.

   2. First object created (date and time): This points to the timestamp of creation of the first object for that archive rule.

      It helps you to roughly identify when the system completed writing the first object based on the installed rule.

   3. Name of the archive

   4. Approximate size of the archive

   5. In Config stipulates that the archive rule is active

   6. In S3 denotes that the archive is present in the object store.

      When a user deletes a rule, the archives that were saved for it persist in archival storage. Even after the rule is no loger active, you can still hydrate the log objects from the existing archive.

   7. Location directory in the archive location

   8. Filters that were used to select the log lines for the archive.

      For example, if the user installed a new archive rule on 2024/09/24 15:00, and started archiving old logs from 2023/01/02 9:00 onwards, then:

      First Archive Record is 2023-01-02 9:00:00 First Object Created is 2024-09-24 15:00:00

To hydrate a specific log, see Hydrate a specific log.

After successful hydration and indexing, the log is inside the Kloudfuse system, and can be reviewed and analyzed like other logs. The user can see the logs from the hydration job by clicking the Logs button and opening a new tab filter set to the job id.

Hydrate a specific log

1. To hydrate specific logs from an archive, hover the cursor over the last column in the table; when the Hydrate button appears, click it.
   Here, we are hydrating the log object bug.
   
   

2. The Hydrating from bug​ interface appears.

3. To hydrate a log, complete these tasks:
   
   

   • Set Time Range

     In Set Time Range (required), select the start and end time. Note that you can specify time measurement either in UTC or in local browser time.

     If you don’t explicitly specify the time range, Kloudfuse uses the default of 1 week after the start of the archive.

     The time selection options are:

     • Absolute time range using the time picker.

     • Quick ranges, such as “Next 5 minutes”, “Next 15 minutes”, and so on, through to “Next 3 days”.

       This assumes the start at the beginning of the first record of the object, and ending calculated by adding the quick range selection.

   • Set filter

     Filter settings are optional.

     Set filter to hydrate log entries that match the specified key/value pairs.

     You can specify multiple filters; Kloudfuse combines them through AND logic. So, a record must match all filters to be hydrated.

     If you don’t specify filters, Kloudfuse potentially hydrates all log records that match the other criteria.

   • Add tags

     Tag settings are optional.

     Set tags to add the same attribute to all hydrated records.

     Specify key/value pairs for each tag. You can specify multiple tags.

     If the log entry already uses a specified key, Kloudfuse overwrites the value attribute.

   • Set Retention

     Set retention (required) to specify how long you want to keep the hydrated records on the Kloudfuse platform. Default retention is 7 days.

4. Click Hydrate to start log hydration.

5. The system notifies you that it started the Hydration job.
   
   

This interface provides a historical overview of all archival jobs by Name, Start Time and End Time, the Status of the job (see Hydration job status), Progress for the currently running job, Filters, Tags, Created at (date and time), and Retention period of the hydrated logs.

Each job also has a unique job id that you can use to locate the hydrated logs in the logs search page. All hydrated logs have the label __kf_hydration_job : <unique_job_id>.

6. After successful hydration and indexing, the log is inside the Kloudfuse system, and can be reviewed and analyzed like other logs.

   The user can see the logs from the hydration job by clicking the Logs button and opening a new tab filter set to the job id.

7. You can cancel a hydration job in progress. Any log lines that have been hydrated before the cancel command remain visible, through the retention time specified in the hydration job.

Hydration job status

The Status of a hydration job is one of the following: