Tue Sep 22 03:49:22 AEST 2026 - Log files ok

 No notable entries in /var/log/messages


Full log: /var/log/messages
<...SKIPPED...>
Sep 22 03:47:22 cephnode03 ceph-mon[2135]: from='mgr.44099 10.4.20.31:0/99874581' entity='mgr.cephnode01.kxbnqe' 
Sep 22 03:47:23 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:47:28 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:47:33 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:47:38 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
<...CURRENT...>
Sep 22 03:47:43 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:47:46 cephnode03 ceph-osd[2024614]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS -------
Sep 22 03:47:46 cephnode03 ceph-osd[2024614]: rocksdb: [db/db_impl/db_impl.cc:1111] #012** DB Stats **#012Uptime(secs): 618000.8 total, 600.0 interval#012Cumulative writes: 133K writes, 508K keys, 133K commit groups, 1.0 writes per commit group, ingest: 0.74 GB, 0.00 MB/s#012Cumulative WAL: 133K writes, 62K syncs, 2.13 writes per sync, written: 0.74 GB, 0.00 MB/s#012Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent#012Interval writes: 142 writes, 550 keys, 142 commit groups, 1.0 writes per commit group, ingest: 0.77 MB, 0.00 MB/s#012Interval WAL: 142 writes, 68 syncs, 2.09 writes per sync, written: 0.00 GB, 0.00 MB/s#012Interval stall: 00:00:0.000 H:M:S, 0.0 percent
Sep 22 03:47:48 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:47:51 cephnode03 systemd[1]: systemd-hostnamed.service: Deactivated successfully.
<...SKIPPED...>
Sep 22 03:48:26 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 
Sep 22 03:48:26 cephnode03 ceph-mon[2135]: cephx server client.crash.cephnode03: handle_request failed to decode CephXAuthenticate: End of buffer [buffer:2]
Sep 22 03:48:26 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: WARNING:ceph-crash:post /var/lib/ceph/crash/2026-09-14T05:59:58.615077Z_79a08790-f583-4f0a-aa30-960b247643d0 as client.crash.cephnode03 failed: 2026-09-21T17:48:26.988+0000 7f73f9739640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:26 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 2026-09-21T17:48:26.988+0000 7f73f3fff640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:26 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 2026-09-21T17:48:26.988+0000 7f73f8f38640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:26 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: [errno 13] RADOS permission denied (error connecting to the cluster)
Sep 22 03:48:26 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 
Sep 22 03:48:27 cephnode03 ceph-mon[2135]: cephx server client.crash.cephnode03: handle_request failed to decode CephXAuthenticate: End of buffer [buffer:2]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: WARNING:ceph-crash:post /var/lib/ceph/crash/2026-09-14T06:01:56.229448Z_bcde091b-ea85-4670-954a-e7efee41f109 as client.crash.cephnode03 failed: 2026-09-21T17:48:27.062+0000 7f9dded76640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 2026-09-21T17:48:27.062+0000 7f9dddd74640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 2026-09-21T17:48:27.062+0000 7f9dde575640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: [errno 13] RADOS permission denied (error connecting to the cluster)
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 
Sep 22 03:48:27 cephnode03 ceph-mon[2135]: cephx server client.crash.cephnode03: handle_request failed to decode CephXAuthenticate: End of buffer [buffer:2]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: WARNING:ceph-crash:post /var/lib/ceph/crash/2026-09-14T12:49:17.865636Z_cfccfb59-a48f-41a9-8961-441cc365eb74 as client.crash.cephnode03 failed: 2026-09-21T17:48:27.136+0000 7f18c37fe640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 2026-09-21T17:48:27.136+0000 7f18c27fc640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 2026-09-21T17:48:27.136+0000 7f18c2ffd640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: [errno 13] RADOS permission denied (error connecting to the cluster)
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 
Sep 22 03:48:27 cephnode03 ceph-mon[2135]: cephx server client.crash.cephnode03: handle_request failed to decode CephXAuthenticate: End of buffer [buffer:2]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: WARNING:ceph-crash:post /var/lib/ceph/crash/2026-09-14T14:07:25.734684Z_51e81f5b-83e4-439d-89f7-453eae36f1a6 as client.crash.cephnode03 failed: 2026-09-21T17:48:27.209+0000 7f9a7bfff640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 2026-09-21T17:48:27.209+0000 7f9a80954640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 2026-09-21T17:48:27.210+0000 7f9a7b7fe640 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: [errno 13] RADOS permission denied (error connecting to the cluster)
Sep 22 03:48:27 cephnode03 ceph-3937760e-9082-11f1-9f3e-1402ec96de8d-crash-cephnode03[2082]: 
Sep 22 03:48:28 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:48:33 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:48:38 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:48:43 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:48:48 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:48:53 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:48:54 cephnode03 systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Sep 22 03:48:58 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:49:02 cephnode03 ceph-osd[17981]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS -------
Sep 22 03:49:02 cephnode03 ceph-osd[17981]: rocksdb: [db/db_impl/db_impl.cc:1111] #012** DB Stats **#012Uptime(secs): 3592802.1 total, 600.0 interval#012Cumulative writes: 729K writes, 2816K keys, 729K commit groups, 1.0 writes per commit group, ingest: 4.46 GB, 0.00 MB/s#012Cumulative WAL: 729K writes, 340K syncs, 2.14 writes per sync, written: 4.46 GB, 0.00 MB/s#012Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent#012Interval writes: 119 writes, 383 keys, 119 commit groups, 1.0 writes per commit group, ingest: 0.60 MB, 0.00 MB/s#012Interval WAL: 119 writes, 53 syncs, 2.25 writes per sync, written: 0.00 GB, 0.00 MB/s#012Interval stall: 00:00:0.000 H:M:S, 0.0 percent
Sep 22 03:49:03 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:49:08 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:49:13 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104
Sep 22 03:49:18 cephnode03 ceph-mon[2135]: mon.cephnode03@2(peon).osd e784 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 348127232 kv_alloc: 318767104