唤醒之后系统卡顿假死

zsrkmyn · 2017-10-18 17:42:17

大概情况是这样的，suspend 一段时间之后唤醒计算机，有概率会出现 load average 狂升的现象，大概可以上升到 20+ 然后彻底死机。键盘和鼠标输入明显延迟，键盘大概延迟 5~10s，无法启动任何新进程，切换 tty2 登录输入密码之后卡住无法登录。感觉像是内存耗尽之后出现的假死。开始以为是 IO 操作阻塞，但是硬盘灯不亮。

用 SysRq 输出 task 信息输出了能有好几十秒，所以基本没法截图了……
输出内存信息如下：

查了下 x86_64 构架中 DMA32 部分是内存 16M~4GB 的地方，还有约 1.5G 剩余，所以大概不是内存耗尽。
输出内存信息的时候看到有 systemd-journald.service 报错信息出现（见上图），怀疑是 systemd-journald 坏了。另外重启之后用 journalctl -b-1 -k -r 查看内核信息，发现只有 suspend 之前的日志，没有唤醒之后的日志。而且两次出现假死都是这样的情况，所以应该就是 systemd-journald 坏了吧……
coredumpctl list 没有发现 systemd-journald 吐的核，所以该怎么确定问题到底在哪呢？

依云 · 2017-10-18 18:45:30

kdump?

也可以调整 console 上的日志级别，把内核日志打过去。

听起来有可能是 fs 坏了啊。

zsrkmyn · 2017-10-19 01:51:49

刚才又发生了一次，其实鼠标没有延迟，键盘延迟以及无法开启新进程更像是某种软件上的阻塞造成的。比如我依然可以用 meta+< 或者 meta+> 在多个工作区正常切换，用 alt+Fx 可以切换 tty，在终端中可以用鼠标正常选择文字等等。但在 shell 中输入字符则有明显延迟，像是输入在 shell 处理字符中阻塞的（感觉因为 shell 不是 line_buffered 的，每个字符都处理）。偶尔可以执行 ps 程序，但是执行之后无法返回 shell，像是是在 shell 输出命令提示符前阻塞了……

调整 console 上的日志级别是指 Alt+SysRq+1~9 吗？这个试过，没有多余输出，而且很奇怪，印象中之前 console 日志级别高的话用 SysRq 会有提示，比如按 i 之后会提示 sending SIGKILL to all processes，按 s 之后会提示 sync finished 之类的等等，现在都完全没有了，按空格也没有出现帮助信息……

如果是 fs 坏掉的话为什么每次都是在 suspend 又唤醒之后发生呢？而且硬盘灯也没亮。一会我先用 live CD fsck 一下看看……

依云 · 2017-10-19 10:54:22

我是说管 fs 的内核线程出事了。

使用内核参数调日志，这样出问题之后你应该能在控制台上看到更多的日志了。

但是 shell 处理单个输入基本不需要 I/O 的呀。难道是 CPU 资源被实时进程占用了？

ps 的结果里有很多 D 状态的进程么？

zsrkmyn · 2017-10-19 12:20:14

昨天仙子说 fs 出现问题的时候确实注意了硬盘问题，但是我说硬盘灯没亮。晚上意识到可能是硬盘就没启动。今早起床又出现了一次这样的问题，这次注意到可能真的是硬盘没有启动，因为硬盘灯连闪烁都没有，即使用 SysRq + s 强制同步也不会有硬盘闪烁。硬盘是否在转倒是没感觉出来（倾向于没在转，因为渣本硬盘转动时有很大震动感和声音），下次再试试。

ps 出来的时候因为在 tty 不能翻太多屏，然而每次挂的时候都开了 chromium，所以 chromium 的进程占据了好大一部分，印象中看起来没有 D，也每太注意，怀疑 systemd-journald 之后没有再想可能是 IO 问题。不过 SysRq + w 好像可以看阻塞任务，今早忘记看了。下次我再注意一下。

zsrkmyn · 2017-10-19 12:31:08

设置 loglevel=9 好像确实输出了更多内核日志。（为什么用 Alt+SysRq+9 就不行呢？）
设置内核参数的时候突然发现上次为了尝试使用 bfq 的时候加的 scsi_mod.use_blk_mq=1 忘记去掉了，怀疑是使用 blk_mq 造成的……

依云 · 2017-10-19 12:47:13

有可能是硬盘没有唤醒。

zsrkmyn · 2017-10-19 18:58:16

仔细感受了一下，硬盘是转了的。SysRq + w 输出了好多阻塞进程。正常状况下应该是没有的……这样的话，硬盘没法 IO，基本所以现象都能解释通了。

在没有开 X 的情况下 suspend 再唤醒，这样可以在唤醒瞬间看到内核日志没有明显异常：

打印阻塞的进程的信息，发现所以被阻塞的进程都阻塞在 schedule 函数里面，截了其中一个如下：

配了 kdump，发生异常后手动触发 crash，保存了 vmcore，还在研究怎么用 _(:з」∠)_

zsrkmyn · 2017-10-19 19:33:16

发现从 vmcore 里可以查询到所有的内核日志，早知道就可以不拍照了……

#
########## 启动部分的日志又臭又长，省略了，下面是开始 suspend 到唤醒刚完毕的日志
#
[   71.859213] PM: Syncing filesystems ... done.
[   72.164858] PM: Preparing system for sleep (mem)
[   72.166237] Freezing user space processes ... (elapsed 0.001 seconds) done.
[   72.168834] OOM killer disabled.
[   72.170112] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[   72.172657] PM: Suspending system (mem)
[   72.173975] Suspending console(s) (use no_console_suspend to debug)
[   72.175552] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[   72.188970] sd 0:0:0:0: [sda] Stopping disk
[   72.584865] PM: suspend of devices complete after 409.461 msecs
[   72.600170] PM: late suspend of devices complete after 15.293 msecs
[   72.616852] PM: noirq suspend of devices complete after 16.674 msecs
[   72.617260] ACPI: Preparing to enter system sleep state S3
[   72.617683] ACPI: EC: event blocked
[   72.617683] ACPI: EC: EC stopped
[   72.617684] PM: Saving platform NVS memory
[   72.617699] Disabling non-boot CPUs ...
[   72.645496] smpboot: CPU 1 is now offline
[   72.664897] smpboot: CPU 2 is now offline
[   72.691581] smpboot: CPU 3 is now offline
[   72.693348] ACPI: Low-level resume complete
[   72.693403] ACPI: EC: EC started
[   72.693403] PM: Restoring platform NVS memory
[   72.693725] Suspended for 2693.522 seconds
[   72.699547] Enabling non-boot CPUs ...
[   72.699581] x86: Booting SMP configuration:
[   72.699582] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   72.702422]  cache: parent cpu1 should not be sleeping
[   72.702753] CPU1 is up
[   72.702820] smpboot: Booting Node 0 Processor 2 APIC 0x2
[   72.706074]  cache: parent cpu2 should not be sleeping
[   72.706454] CPU2 is up
[   72.706501] smpboot: Booting Node 0 Processor 3 APIC 0x3
[   72.709386]  cache: parent cpu3 should not be sleeping
[   72.709938] CPU3 is up
[   72.713349] ACPI: Waking up from system sleep state S3
[   72.732022] PM: noirq resume of devices complete after 18.198 msecs
[   72.732610] PM: early resume of devices complete after 0.558 msecs
[   72.732739] ACPI: EC: event unblocked
[   72.734005] ath: phy0: ASPM enabled: 0x42
[   72.734249] ACPI: button: The lid device is not compliant to SW_LID.
[   72.743129] sd 0:0:0:0: [sda] Starting disk
[   72.828326] r8169 0000:04:00.2 enp4s0f2: link down
[   72.968529] usb 1-1.1: reset high-speed USB device number 3 using ehci-pci
[   73.066158] ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   73.067891] ata6.00: configured for UDMA/100
[   73.123414] PM: resume of devices complete after 390.792 msecs
[   73.178794] PM: Finishing wakeup.
[   73.179843] OOM killer enabled.
[   73.180882] Restarting tasks ... done.
[   73.456535] psmouse serio2: synaptics: queried max coordinates: x [..5774], y [..5028]
[   73.486164] psmouse serio2: synaptics: queried min coordinates: x [1168..], y [880..]
[   74.515913] r8169 0000:04:00.2 enp4s0f2: link up
[   75.331876] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   75.359171] ata1.00: configured for UDMA/133
#
########## 下面唤醒一段时间后的输出
#
[  244.985485] INFO: task kworker/u16:7:159 blocked for more than 120 seconds.
[  244.987914]       Not tainted 4.13.5-1-lily #12
[  244.990313] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  244.992792] kworker/u16:7   D    0   159      2 0x00000000
[  244.995244] Workqueue: writeback wb_workfn (flush-8:0)
[  244.997705] Call Trace:
[  244.998641]  __schedule+0x239/0x890
[  244.999530]  schedule+0x3d/0x90
[  245.000400]  io_schedule+0x16/0x40
[  245.001269]  wbt_wait+0x1c6/0x350
[  245.002144]  ? wait_woken+0x80/0x80
[  245.003020]  blk_mq_make_request+0xe2/0x6c0
[  245.003907]  generic_make_request+0x125/0x320
[  245.004790]  submit_bio+0x73/0x150
[  245.005689]  ? submit_bio+0x73/0x150
[  245.006591]  ext4_io_submit+0x4c/0x60 [ext4]
[  245.007490]  ext4_bio_write_page+0x252/0x510 [ext4]
[  245.008394]  mpage_submit_page+0x59/0x80 [ext4]
[  245.009302]  mpage_process_page_bufs+0xf6/0x100 [ext4]
[  245.010205]  mpage_prepare_extent_to_map+0x1e9/0x300 [ext4]
[  245.011111]  ext4_writepages+0x446/0xf50 [ext4]
[  245.012010]  ? update_load_avg+0x426/0x5a0
[  245.012896]  ? __update_load_avg_se.isra.36+0x155/0x160
[  245.013786]  ? update_load_avg+0x426/0x5a0
[  245.014666]  ? update_curr+0xf5/0x1e0
[  245.015540]  ? __enqueue_entity+0x6c/0x70
[  245.016409]  ? enqueue_entity+0x109/0x7b0
[  245.017270]  do_writepages+0x48/0xd0
[  245.018121]  ? do_writepages+0x48/0xd0
[  245.018983]  ? check_preempt_curr+0x2d/0x90
[  245.019837]  ? ttwu_do_wakeup+0x1e/0x160
[  245.020682]  ? ttwu_do_activate+0x7a/0x90
[  245.021523]  __writeback_single_inode+0x45/0x390
[  245.022368]  ? __writeback_single_inode+0x45/0x390
[  245.023198]  writeback_sb_inodes+0x201/0x510
[  245.024024]  __writeback_inodes_wb+0x67/0xb0
[  245.024835]  wb_writeback+0x29b/0x340
[  245.025647]  ? find_next_bit+0xb/0x10
[  245.026435]  wb_workfn+0x171/0x3c0
[  245.027214]  ? wb_workfn+0x171/0x3c0
[  245.027987]  process_one_work+0x1de/0x430
[  245.028768]  worker_thread+0x47/0x3f0
[  245.029546]  kthread+0x125/0x140
[  245.030302]  ? process_one_work+0x430/0x430
[  245.031059]  ? kthread_create_on_node+0x70/0x70
[  245.031803]  ret_from_fork+0x25/0x30
[  245.032547] INFO: task jbd2/sda2-8:191 blocked for more than 120 seconds.
[  245.033288]       Not tainted 4.13.5-1-lily #12
[  245.034021] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  245.034770] jbd2/sda2-8     D    0   191      2 0x00000000
[  245.035513] Call Trace:
[  245.036275]  __schedule+0x239/0x890
[  245.038309]  ? blk_mq_sched_insert_requests+0x94/0x130
[  245.040334]  ? bit_wait+0x60/0x60
[  245.042347]  schedule+0x3d/0x90
[  245.044343]  io_schedule+0x16/0x40
[  245.046360]  bit_wait_io+0x11/0x60
[  245.048357]  __wait_on_bit+0x31/0x90
[  245.049168]  out_of_line_wait_on_bit+0x90/0xb0
[  245.049895]  ? bit_waitqueue+0x40/0x40
[  245.050619]  __wait_on_buffer+0x28/0x30
[  245.051346]  jbd2_journal_commit_transaction+0xf66/0x1820 [jbd2]
[  245.052090]  ? finish_task_switch+0x75/0x200
[  245.052836]  kjournald2+0xd5/0x270 [jbd2]
[  245.053570]  ? kjournald2+0xd5/0x270 [jbd2]
[  245.054305]  ? wait_woken+0x80/0x80
[  245.055029]  kthread+0x125/0x140
[  245.055753]  ? commit_timeout+0x10/0x10 [jbd2]
[  245.056473]  ? kthread_create_on_node+0x70/0x70
[  245.057190]  ret_from_fork+0x25/0x30
[  245.057905] INFO: task jbd2/sda8-8:311 blocked for more than 120 seconds.
[  245.058635]       Not tainted 4.13.5-1-lily #12
[  245.059376] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  245.060142] jbd2/sda8-8     D    0   311      2 0x00000000
[  245.060909] Call Trace:
[  245.061671]  __schedule+0x239/0x890
[  245.062436]  ? blk_mq_sched_insert_requests+0x94/0x130
[  245.063211]  ? bit_wait+0x60/0x60
[  245.063984]  schedule+0x3d/0x90
[  245.064752]  io_schedule+0x16/0x40
[  245.065525]  bit_wait_io+0x11/0x60
[  245.066296]  __wait_on_bit+0x31/0x90
[  245.068431]  out_of_line_wait_on_bit+0x90/0xb0
[  245.070596]  ? bit_waitqueue+0x40/0x40
[  245.072769]  __wait_on_buffer+0x28/0x30
[  245.074921]  jbd2_journal_commit_transaction+0xf66/0x1820 [jbd2]
[  245.077113]  ? finish_task_switch+0x75/0x200
[  245.079333]  kjournald2+0xd5/0x270 [jbd2]
[  245.080210]  ? kjournald2+0xd5/0x270 [jbd2]
[  245.080997]  ? wait_woken+0x80/0x80
[  245.081780]  kthread+0x125/0x140
[  245.082566]  ? commit_timeout+0x10/0x10 [jbd2]
[  245.083357]  ? kthread_create_on_node+0x70/0x70
[  245.084148]  ret_from_fork+0x25/0x30
[  245.085331] INFO: task (p-helper):514 blocked for more than 120 seconds.
[  245.086513]       Not tainted 4.13.5-1-lily #12
[  245.087670] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  245.088857] (p-helper)      D    0   514    411 0x00000000
[  245.090045] Call Trace:
[  245.091222]  __schedule+0x239/0x890
[  245.092399]  ? bit_wait+0x60/0x60
[  245.093562]  schedule+0x3d/0x90
[  245.094724]  io_schedule+0x16/0x40
[  245.095531]  bit_wait_io+0x11/0x60
[  245.096338]  __wait_on_bit+0x31/0x90
[  245.098283]  ? bio_alloc_bioset+0xcc/0x1e0
[  245.100250]  out_of_line_wait_on_bit+0x90/0xb0
[  245.102210]  ? bit_waitqueue+0x40/0x40
[  245.104152]  __wait_on_buffer+0x28/0x30
[  245.106157]  __ext4_get_inode_loc+0x2f8/0x430 [ext4]
[  245.107908]  ext4_iget+0x8e/0xbd0 [ext4]
[  245.109668]  ? legitimize_path.isra.28+0x2e/0x60
[  245.111604]  ext4_iget_normal+0x30/0x40 [ext4]
[  245.113579]  ext4_lookup+0x138/0x260 [ext4]
[  245.115518]  path_openat+0xc05/0x1400
[  245.117459]  do_filp_open+0x9b/0x110
[  245.119394]  ? do_wp_page+0x159/0x4a0
[  245.121302]  ? __slab_alloc.isra.74+0x2b/0x40
[  245.122144]  ? sysctl_head_finish.part.22+0x43/0x50
[  245.122904]  do_open_execat+0x7c/0x190
[  245.123646]  ? do_open_execat+0x7c/0x190
[  245.124375]  do_execveat_common.isra.36+0x1db/0x830
[  245.125110]  ? strncpy_from_user+0x4d/0x170
[  245.125854]  ? getname_flags+0x10/0x190
[  245.126588]  SyS_execve+0x31/0x40
[  245.127312]  do_syscall_64+0x5b/0xc0
[  245.128038]  entry_SYSCALL64_slow_path+0x25/0x25
[  245.128775] RIP: 0033:0x7ff1e8fba5e7
[  245.129511] RSP: 002b:00007ffd35ce4428 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
[  245.130274] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007ff1e8fba5e7
[  245.131040] RDX: 000055824ecc3e80 RSI: 000055824ecf89d0 RDI: 000055824ece3d10
[  245.131813] RBP: 00007ffd35ce47c0 R08: 0000000000000000 R09: 0000000000000001
[  245.132603] R10: 00007ff1e92a7af8 R11: 0000000000000246 R12: 000055824ece53b0
[  245.133375] R13: 000055824ecf89d0 R14: 000055824ece54a0 R15: 000055824ece5020
[  245.134145] INFO: task (p-helper):517 blocked for more than 120 seconds.
[  245.134923]       Not tainted 4.13.5-1-lily #12
[  245.135706] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  245.136553] (p-helper)      D    0   517    411 0x00000000
[  245.137386] Call Trace:
[  245.138603]  __schedule+0x239/0x890
[  245.139825]  schedule+0x3d/0x90
[  245.141032]  d_alloc_parallel+0x491/0x5a0
[  245.142227]  ? __d_lookup+0x86/0x150
[  245.143396]  ? wake_up_q+0x80/0x80
[  245.144539]  path_openat+0xdaa/0x1400
[  245.145667]  ? next_arg+0xd0/0x110
[  245.146785]  ? unlock_page_memcg+0x12/0x20
[  245.147909]  do_filp_open+0x9b/0x110
[  245.148693]  ? do_wp_page+0x159/0x4a0
[  245.149450]  ? __handle_mm_fault+0xab2/0x1000
[  245.150211]  ? sysctl_head_finish.part.22+0x43/0x50
[  245.151357]  do_open_execat+0x7c/0x190
[  245.152516]  ? do_open_execat+0x7c/0x190
[  245.153659]  do_execveat_common.isra.36+0x1db/0x830
[  245.154807]  ? strncpy_from_user+0x4d/0x170
[  245.155961]  ? getname_flags+0x10/0x190
[  245.157107]  SyS_execve+0x31/0x40
[  245.158245]  do_syscall_64+0x5b/0xc0
[  245.159379]  entry_SYSCALL64_slow_path+0x25/0x25
[  245.160526] RIP: 0033:0x7ff1e8fba5e7
[  245.161316] RSP: 002b:00007ffd35ce4428 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
[  245.162502] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007ff1e8fba5e7
[  245.163684] RDX: 000055824ecf8e10 RSI: 000055824ece9e30 RDI: 000055824ece5e10
[  245.164878] RBP: 00007ffd35ce47c0 R08: 0000000000000000 R09: 0000000000000000
[  245.166084] R10: 000055824ecea290 R11: 0000000000000246 R12: 000055824ece7170
[  245.167290] R13: 000055824ece9e30 R14: 000055824ece7260 R15: 000055824ece6de0
[  282.079301] systemd[1]: systemd-journald.service: State 'stop-sigabrt' timed out. Terminating.
[  367.861684] INFO: task kworker/u16:7:159 blocked for more than 120 seconds.
[  367.864067]       Not tainted 4.13.5-1-lily #12
[  367.866438] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  367.868879] kworker/u16:7   D    0   159      2 0x00000000
[  367.871354] Workqueue: writeback wb_workfn (flush-8:0)
[  367.873840] Call Trace:
[  367.874813]  __schedule+0x239/0x890
[  367.875689]  schedule+0x3d/0x90
[  367.876547]  io_schedule+0x16/0x40
[  367.877398]  wbt_wait+0x1c6/0x350
[  367.878243]  ? wait_woken+0x80/0x80
[  367.879084]  blk_mq_make_request+0xe2/0x6c0
[  367.879925]  generic_make_request+0x125/0x320
[  367.880753]  submit_bio+0x73/0x150
[  367.881572]  ? submit_bio+0x73/0x150
[  367.882389]  ext4_io_submit+0x4c/0x60 [ext4]
[  367.883191]  ext4_bio_write_page+0x252/0x510 [ext4]
[  367.883993]  mpage_submit_page+0x59/0x80 [ext4]
[  367.884795]  mpage_process_page_bufs+0xf6/0x100 [ext4]
[  367.885608]  mpage_prepare_extent_to_map+0x1e9/0x300 [ext4]
[  367.886512]  ext4_writepages+0x446/0xf50 [ext4]
[  367.888600]  ? update_load_avg+0x426/0x5a0
[  367.890873]  ? __update_load_avg_se.isra.36+0x155/0x160
[  367.893162]  ? update_load_avg+0x426/0x5a0
[  367.895505]  ? update_curr+0xf5/0x1e0
[  367.897803]  ? __enqueue_entity+0x6c/0x70
[  367.900095]  ? enqueue_entity+0x109/0x7b0
[  367.900995]  do_writepages+0x48/0xd0
[  367.901825]  ? do_writepages+0x48/0xd0
[  367.902641]  ? check_preempt_curr+0x2d/0x90
[  367.903451]  ? ttwu_do_wakeup+0x1e/0x160
[  367.904260]  ? ttwu_do_activate+0x7a/0x90
[  367.905073]  __writeback_single_inode+0x45/0x390
[  367.905880]  ? __writeback_single_inode+0x45/0x390
[  367.906688]  writeback_sb_inodes+0x201/0x510
[  367.907500]  __writeback_inodes_wb+0x67/0xb0
[  367.908326]  wb_writeback+0x29b/0x340
[  367.909138]  ? find_next_bit+0xb/0x10
[  367.909943]  wb_workfn+0x171/0x3c0
[  367.910734]  ? wb_workfn+0x171/0x3c0
[  367.911514]  process_one_work+0x1de/0x430
[  367.912288]  worker_thread+0x47/0x3f0
[  367.913054]  kthread+0x125/0x140
[  367.913813]  ? process_one_work+0x430/0x430
[  367.914570]  ? kthread_create_on_node+0x70/0x70
[  367.915331]  ret_from_fork+0x25/0x30
[  367.916080] INFO: task jbd2/sda2-8:191 blocked for more than 120 seconds.
[  367.916830]       Not tainted 4.13.5-1-lily #12
[  367.917562] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  367.918316] jbd2/sda2-8     D    0   191      2 0x00000000
[  367.919061] Call Trace:
[  367.919804]  __schedule+0x239/0x890
[  367.920521]  ? blk_mq_sched_insert_requests+0x94/0x130
[  367.921243]  ? bit_wait+0x60/0x60
[  367.921970]  schedule+0x3d/0x90
[  367.922685]  io_schedule+0x16/0x40
[  367.923401]  bit_wait_io+0x11/0x60
[  367.924115]  __wait_on_bit+0x31/0x90
[  367.924831]  out_of_line_wait_on_bit+0x90/0xb0
[  367.925558]  ? bit_waitqueue+0x40/0x40
[  367.926278]  __wait_on_buffer+0x28/0x30
[  367.926997]  jbd2_journal_commit_transaction+0xf66/0x1820 [jbd2]
[  367.927728]  ? finish_task_switch+0x75/0x200
[  367.928471]  kjournald2+0xd5/0x270 [jbd2]
[  367.929210]  ? kjournald2+0xd5/0x270 [jbd2]
[  367.929935]  ? wait_woken+0x80/0x80
[  367.930654]  kthread+0x125/0x140
[  367.931366]  ? commit_timeout+0x10/0x10 [jbd2]
[  367.932091]  ? kthread_create_on_node+0x70/0x70
[  367.932803]  ret_from_fork+0x25/0x30
[  367.933513] INFO: task jbd2/sda8-8:311 blocked for more than 120 seconds.
[  367.934235]       Not tainted 4.13.5-1-lily #12
[  367.934978] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  367.935738] jbd2/sda8-8     D    0   311      2 0x00000000
[  367.936499] Call Trace:
[  367.937257]  __schedule+0x239/0x890
[  367.938016]  ? blk_mq_sched_insert_requests+0x94/0x130
[  367.938794]  ? bit_wait+0x60/0x60
[  367.939560]  schedule+0x3d/0x90
[  367.940328]  io_schedule+0x16/0x40
[  367.941090]  bit_wait_io+0x11/0x60
[  367.941858]  __wait_on_bit+0x31/0x90
[  367.942611]  out_of_line_wait_on_bit+0x90/0xb0
[  367.943369]  ? bit_waitqueue+0x40/0x40
[  367.944123]  __wait_on_buffer+0x28/0x30
[  367.944883]  jbd2_journal_commit_transaction+0xf66/0x1820 [jbd2]
[  367.945660]  ? finish_task_switch+0x75/0x200
[  367.946438]  kjournald2+0xd5/0x270 [jbd2]
[  367.947214]  ? kjournald2+0xd5/0x270 [jbd2]
[  367.947990]  ? wait_woken+0x80/0x80
[  367.948774]  kthread+0x125/0x140
[  367.949559]  ? commit_timeout+0x10/0x10 [jbd2]
[  367.950333]  ? kthread_create_on_node+0x70/0x70
[  367.951105]  ret_from_fork+0x25/0x30
[  367.951883] INFO: task (p-helper):514 blocked for more than 120 seconds.
[  367.952662]       Not tainted 4.13.5-1-lily #12
[  367.953424] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  367.954200] (p-helper)      D    0   514    411 0x00000000
[  367.954991] Call Trace:
[  367.955769]  __schedule+0x239/0x890
[  367.956544]  ? bit_wait+0x60/0x60
[  367.957312]  schedule+0x3d/0x90
[  367.958080]  io_schedule+0x16/0x40
[  367.958855]  bit_wait_io+0x11/0x60
[  367.959621]  __wait_on_bit+0x31/0x90
[  367.960388]  ? bio_alloc_bioset+0xcc/0x1e0
[  367.961157]  out_of_line_wait_on_bit+0x90/0xb0
[  367.961939]  ? bit_waitqueue+0x40/0x40
[  367.962709]  __wait_on_buffer+0x28/0x30
[  367.963478]  __ext4_get_inode_loc+0x2f8/0x430 [ext4]
[  367.964246]  ext4_iget+0x8e/0xbd0 [ext4]
[  367.965011]  ? legitimize_path.isra.28+0x2e/0x60
[  367.965785]  ext4_iget_normal+0x30/0x40 [ext4]
[  367.966560]  ext4_lookup+0x138/0x260 [ext4]
[  367.967328]  path_openat+0xc05/0x1400
[  367.968093]  do_filp_open+0x9b/0x110
[  367.968875]  ? do_wp_page+0x159/0x4a0
[  367.970027]  ? __slab_alloc.isra.74+0x2b/0x40
[  367.971157]  ? sysctl_head_finish.part.22+0x43/0x50
[  367.972288]  do_open_execat+0x7c/0x190
[  367.973386]  ? do_open_execat+0x7c/0x190
[  367.974463]  do_execveat_common.isra.36+0x1db/0x830
[  367.975563]  ? strncpy_from_user+0x4d/0x170
[  367.976538]  ? getname_flags+0x10/0x190
[  367.977495]  SyS_execve+0x31/0x40
[  367.978453]  do_syscall_64+0x5b/0xc0
[  367.979410]  entry_SYSCALL64_slow_path+0x25/0x25
[  367.980376] RIP: 0033:0x7ff1e8fba5e7
[  367.981343] RSP: 002b:00007ffd35ce4428 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
[  367.982396] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007ff1e8fba5e7
[  367.983569] RDX: 000055824ecc3e80 RSI: 000055824ecf89d0 RDI: 000055824ece3d10
[  367.985035] RBP: 00007ffd35ce47c0 R08: 0000000000000000 R09: 0000000000000001
[  367.986487] R10: 00007ff1e92a7af8 R11: 0000000000000246 R12: 000055824ece53b0
[  367.987932] R13: 000055824ecf89d0 R14: 000055824ece54a0 R15: 000055824ece5020
[  367.989391] INFO: task (p-helper):517 blocked for more than 120 seconds.
[  367.990843]       Not tainted 4.13.5-1-lily #12
[  367.992299] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  367.993802] (p-helper)      D    0   517    411 0x00000000
[  367.995346] Call Trace:
[  367.996218]  __schedule+0x239/0x890
[  367.997028]  schedule+0x3d/0x90
[  367.997833]  d_alloc_parallel+0x491/0x5a0
[  367.998633]  ? __d_lookup+0x86/0x150
[  367.999423]  ? wake_up_q+0x80/0x80
[  368.001539]  path_openat+0xdaa/0x1400
[  368.003674]  ? next_arg+0xd0/0x110
[  368.005755]  ? unlock_page_memcg+0x12/0x20
[  368.007853]  do_filp_open+0x9b/0x110
[  368.009958]  ? do_wp_page+0x159/0x4a0
[  368.012070]  ? __handle_mm_fault+0xab2/0x1000
[  368.012957]  ? sysctl_head_finish.part.22+0x43/0x50
[  368.014685]  do_open_execat+0x7c/0x190
[  368.016432]  ? do_open_execat+0x7c/0x190
[  368.018352]  do_execveat_common.isra.36+0x1db/0x830
[  368.020245]  ? strncpy_from_user+0x4d/0x170
[  368.022137]  ? getname_flags+0x10/0x190
[  368.024293]  SyS_execve+0x31/0x40
[  368.026467]  do_syscall_64+0x5b/0xc0
[  368.028182]  entry_SYSCALL64_slow_path+0x25/0x25
[  368.029908] RIP: 0033:0x7ff1e8fba5e7
[  368.031612] RSP: 002b:00007ffd35ce4428 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
[  368.033363] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007ff1e8fba5e7
[  368.035186] RDX: 000055824ecf8e10 RSI: 000055824ece9e30 RDI: 000055824ece5e10
[  368.036980] RBP: 00007ffd35ce47c0 R08: 0000000000000000 R09: 0000000000000000
[  368.037861] R10: 000055824ecea290 R11: 0000000000000246 R12: 000055824ece7170
[  368.038680] R13: 000055824ece9e30 R14: 000055824ece7260 R15: 000055824ece6de0
[  372.329932] systemd[1]: systemd-journald.service: State 'stop-sigterm' timed out. Killing.
[  372.335013] systemd[1]: systemd-journald.service: Killing process 220 (systemd-journal) with signal SIGKILL.
[  372.339244] systemd[1]: systemd-journald.service: Main process exited, code=killed, status=9/KILL
[  372.341734] systemd[1]: systemd-journald.service: Failed with result 'watchdog'.
[  372.343362] systemd[1]: systemd-journald.service: Service has no hold-off time, scheduling restart.
[  372.344331] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.
[  372.345576] systemd[1]: Stopped Flush Journal to Persistent Storage.
[  372.346475] systemd[1]: Stopping Flush Journal to Persistent Storage...
[  372.347359] systemd[1]: Stopped Journal Service.
[  372.349232] systemd[1]: Starting Journal Service...
[  462.580615] systemd[1]: systemd-journald.service: Start operation timed out. Terminating.
#
########## 此处按下 SysRq + w 输出阻塞进程信息
#
[  465.305411] sysrq: SysRq : Show Blocked State
[  465.305422]   task                        PC stack   pid father
[  465.307980] kworker/u16:7   D    0   159      2 0x00000000
[  465.310409] Workqueue: writeback wb_workfn (flush-8:0)
[  465.312850] Call Trace:
[  465.315252]  __schedule+0x239/0x890
[  465.317622]  schedule+0x3d/0x90
[  465.319970]  io_schedule+0x16/0x40
[  465.322252]  wbt_wait+0x1c6/0x350
[  465.324485]  ? wait_woken+0x80/0x80
[  465.326724]  blk_mq_make_request+0xe2/0x6c0
[  465.328969]  generic_make_request+0x125/0x320
[  465.331214]  submit_bio+0x73/0x150
[  465.333473]  ? submit_bio+0x73/0x150
[  465.335756]  ext4_io_submit+0x4c/0x60 [ext4]
[  465.338041]  ext4_bio_write_page+0x252/0x510 [ext4]
[  465.340366]  mpage_submit_page+0x59/0x80 [ext4]
[  465.342666]  mpage_process_page_bufs+0xf6/0x100 [ext4]
[  465.344977]  mpage_prepare_extent_to_map+0x1e9/0x300 [ext4]
[  465.347328]  ext4_writepages+0x446/0xf50 [ext4]
[  465.349641]  ? update_load_avg+0x426/0x5a0
[  465.351956]  ? __update_load_avg_se.isra.36+0x155/0x160
[  465.354310]  ? update_load_avg+0x426/0x5a0
[  465.356624]  ? update_curr+0xf5/0x1e0
[  465.358926]  ? __enqueue_entity+0x6c/0x70
[  465.361246]  ? enqueue_entity+0x109/0x7b0
[  465.363537]  do_writepages+0x48/0xd0
[  465.365821]  ? do_writepages+0x48/0xd0
[  465.368131]  ? check_preempt_curr+0x2d/0x90
[  465.370439]  ? ttwu_do_wakeup+0x1e/0x160
[  465.372743]  ? ttwu_do_activate+0x7a/0x90
[  465.375010]  __writeback_single_inode+0x45/0x390
[  465.377263]  ? __writeback_single_inode+0x45/0x390
[  465.379513]  writeback_sb_inodes+0x201/0x510
[  465.381723]  __writeback_inodes_wb+0x67/0xb0
[  465.383909]  wb_writeback+0x29b/0x340
[  465.386106]  ? find_next_bit+0xb/0x10
[  465.388280]  wb_workfn+0x171/0x3c0
[  465.390401]  ? wb_workfn+0x171/0x3c0
[  465.392494]  process_one_work+0x1de/0x430
[  465.394587]  worker_thread+0x47/0x3f0
[  465.396637]  kthread+0x125/0x140
[  465.398656]  ? process_one_work+0x430/0x430
[  465.400694]  ? kthread_create_on_node+0x70/0x70
[  465.402677]  ret_from_fork+0x25/0x30
[  465.404647] jbd2/sda2-8     D    0   191      2 0x00000000
[  465.406652] Call Trace:
[  465.408620]  __schedule+0x239/0x890
[  465.410555]  ? blk_mq_sched_insert_requests+0x94/0x130
[  465.412468]  ? bit_wait+0x60/0x60
[  465.414377]  schedule+0x3d/0x90
[  465.416241]  io_schedule+0x16/0x40
[  465.418089]  bit_wait_io+0x11/0x60
[  465.419953]  __wait_on_bit+0x31/0x90
[  465.421801]  out_of_line_wait_on_bit+0x90/0xb0
[  465.423668]  ? bit_waitqueue+0x40/0x40
[  465.425537]  __wait_on_buffer+0x28/0x30
[  465.427441]  jbd2_journal_commit_transaction+0xf66/0x1820 [jbd2]
[  465.429359]  ? finish_task_switch+0x75/0x200
[  465.431283]  kjournald2+0xd5/0x270 [jbd2]
[  465.433221]  ? kjournald2+0xd5/0x270 [jbd2]
[  465.435143]  ? wait_woken+0x80/0x80
[  465.437051]  kthread+0x125/0x140
[  465.438953]  ? commit_timeout+0x10/0x10 [jbd2]
[  465.440879]  ? kthread_create_on_node+0x70/0x70
[  465.442785]  ret_from_fork+0x25/0x30
[  465.444685] jbd2/sda8-8     D    0   311      2 0x00000000
[  465.446615] Call Trace:
[  465.448493]  __schedule+0x239/0x890
[  465.450367]  ? blk_mq_sched_insert_requests+0x94/0x130
[  465.452259]  ? bit_wait+0x60/0x60
[  465.454167]  schedule+0x3d/0x90
[  465.456051]  io_schedule+0x16/0x40
[  465.457934]  bit_wait_io+0x11/0x60
[  465.459831]  __wait_on_bit+0x31/0x90
[  465.461712]  out_of_line_wait_on_bit+0x90/0xb0
[  465.463609]  ? bit_waitqueue+0x40/0x40
[  465.465501]  __wait_on_buffer+0x28/0x30
[  465.467429]  jbd2_journal_commit_transaction+0xf66/0x1820 [jbd2]
[  465.469373]  ? finish_task_switch+0x75/0x200
[  465.471316]  kjournald2+0xd5/0x270 [jbd2]
[  465.473272]  ? kjournald2+0xd5/0x270 [jbd2]
[  465.475208]  ? wait_woken+0x80/0x80
[  465.477135]  kthread+0x125/0x140
[  465.479055]  ? commit_timeout+0x10/0x10 [jbd2]
[  465.481014]  ? kthread_create_on_node+0x70/0x70
[  465.482962]  ret_from_fork+0x25/0x30
[  465.484928] kworker/u16:63  D    0   507      2 0x00000000
[  465.486934] Workqueue: events_unbound async_run_entry_fn
[  465.488937] Call Trace:
[  465.490921]  __schedule+0x239/0x890
[  465.492926]  schedule+0x3d/0x90
[  465.494891]  schedule_timeout+0x180/0x390
[  465.496850]  ? call_timer_fn+0x160/0x160
[  465.498796]  ? __blk_mq_run_hw_queue+0x86/0x90
[  465.500777]  io_schedule_timeout+0x1e/0x50
[  465.502731]  ? io_schedule_timeout+0x1e/0x50
[  465.504676]  wait_for_completion_io_timeout+0x9e/0x130
[  465.506669]  ? wake_up_q+0x80/0x80
[  465.508648]  blk_execute_rq+0x6f/0xa0
[  465.510650]  scsi_execute+0xe3/0x250 [scsi_mod]
[  465.512678]  sd_start_stop_device+0x98/0x160 [sd_mod]
[  465.514702]  sd_resume+0x58/0xa0 [sd_mod]
[  465.516731]  do_scsi_resume+0x1b/0x20 [scsi_mod]
[  465.518775]  scsi_dev_type_resume+0x29/0x90 [scsi_mod]
[  465.520833]  async_sdev_resume+0x15/0x20 [scsi_mod]
[  465.522854]  async_run_entry_fn+0x36/0x150
[  465.524869]  process_one_work+0x1de/0x430
[  465.526903]  worker_thread+0x47/0x3f0
[  465.528913]  kthread+0x125/0x140
[  465.530916]  ? process_one_work+0x430/0x430
[  465.532941]  ? kthread_create_on_node+0x70/0x70
[  465.534958]  ret_from_fork+0x25/0x30
[  465.536965] (p-helper)      D    0   514    411 0x00000000
[  465.538995] Call Trace:
[  465.541012]  __schedule+0x239/0x890
[  465.543024]  ? bit_wait+0x60/0x60
[  465.545016]  schedule+0x3d/0x90
[  465.547012]  io_schedule+0x16/0x40
[  465.548991]  bit_wait_io+0x11/0x60
[  465.550964]  __wait_on_bit+0x31/0x90
[  465.552969]  ? bio_alloc_bioset+0xcc/0x1e0
[  465.554958]  out_of_line_wait_on_bit+0x90/0xb0
[  465.556957]  ? bit_waitqueue+0x40/0x40
[  465.558946]  __wait_on_buffer+0x28/0x30
[  465.560981]  __ext4_get_inode_loc+0x2f8/0x430 [ext4]
[  465.563001]  ext4_iget+0x8e/0xbd0 [ext4]
[  465.564969]  ? legitimize_path.isra.28+0x2e/0x60
[  465.566993]  ext4_iget_normal+0x30/0x40 [ext4]
[  465.569004]  ext4_lookup+0x138/0x260 [ext4]
[  465.570985]  path_openat+0xc05/0x1400
[  465.572989]  do_filp_open+0x9b/0x110
[  465.574957]  ? do_wp_page+0x159/0x4a0
[  465.576936]  ? __slab_alloc.isra.74+0x2b/0x40
[  465.578903]  ? sysctl_head_finish.part.22+0x43/0x50
[  465.580888]  do_open_execat+0x7c/0x190
[  465.582833]  ? do_open_execat+0x7c/0x190
[  465.584769]  do_execveat_common.isra.36+0x1db/0x830
[  465.586751]  ? strncpy_from_user+0x4d/0x170
[  465.588721]  ? getname_flags+0x10/0x190
[  465.590688]  SyS_execve+0x31/0x40
[  465.592667]  do_syscall_64+0x5b/0xc0
[  465.594625]  entry_SYSCALL64_slow_path+0x25/0x25
[  465.596590] RIP: 0033:0x7ff1e8fba5e7
[  465.598547] RSP: 002b:00007ffd35ce4428 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
[  465.600596] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007ff1e8fba5e7
[  465.602633] RDX: 000055824ecc3e80 RSI: 000055824ecf89d0 RDI: 000055824ece3d10
[  465.604694] RBP: 00007ffd35ce47c0 R08: 0000000000000000 R09: 0000000000000001
[  465.606786] R10: 00007ff1e92a7af8 R11: 0000000000000246 R12: 000055824ece53b0
[  465.608852] R13: 000055824ecf89d0 R14: 000055824ece54a0 R15: 000055824ece5020
[  465.610925] (p-helper)      D    0   517    411 0x00000000
[  465.613021] Call Trace:
[  465.615064]  __schedule+0x239/0x890
[  465.617100]  schedule+0x3d/0x90
[  465.619126]  d_alloc_parallel+0x491/0x5a0
[  465.621184]  ? __d_lookup+0x86/0x150
[  465.623211]  ? wake_up_q+0x80/0x80
[  465.625237]  path_openat+0xdaa/0x1400
[  465.627282]  ? next_arg+0xd0/0x110
[  465.629295]  ? unlock_page_memcg+0x12/0x20
[  465.631310]  do_filp_open+0x9b/0x110
[  465.633317]  ? do_wp_page+0x159/0x4a0
[  465.635294]  ? __handle_mm_fault+0xab2/0x1000
[  465.637288]  ? sysctl_head_finish.part.22+0x43/0x50
[  465.639323]  do_open_execat+0x7c/0x190
[  465.641337]  ? do_open_execat+0x7c/0x190
[  465.643365]  do_execveat_common.isra.36+0x1db/0x830
[  465.645418]  ? strncpy_from_user+0x4d/0x170
[  465.647496]  ? getname_flags+0x10/0x190
[  465.649555]  SyS_execve+0x31/0x40
[  465.651595]  do_syscall_64+0x5b/0xc0
[  465.653654]  entry_SYSCALL64_slow_path+0x25/0x25
[  465.655701] RIP: 0033:0x7ff1e8fba5e7
[  465.657736] RSP: 002b:00007ffd35ce4428 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
[  465.659860] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007ff1e8fba5e7
[  465.661988] RDX: 000055824ecf8e10 RSI: 000055824ece9e30 RDI: 000055824ece5e10
[  465.664126] RBP: 00007ffd35ce47c0 R08: 0000000000000000 R09: 0000000000000000
[  465.666292] R10: 000055824ecea290 R11: 0000000000000246 R12: 000055824ece7170
[  465.668449] R13: 000055824ece9e30 R14: 000055824ece7260 R15: 000055824ece6de0
[  465.670616] pppd            D    0   525      1 0x00000004
[  465.672820] Call Trace:
[  465.674981]  __schedule+0x239/0x890
[  465.677136]  schedule+0x3d/0x90
[  465.679296]  io_schedule+0x16/0x40
[  465.681436]  generic_file_read_iter+0x344/0x9d0
[  465.683572]  ? page_cache_tree_insert+0xc0/0xc0
[  465.685730]  ext4_file_read_iter+0x3e/0x40 [ext4]
[  465.687894]  __vfs_read+0xed/0x140
[  465.690033]  vfs_read+0x8e/0x130
[  465.692166]  prepare_binprm+0x110/0x1f0
[  465.694313]  do_execveat_common.isra.36+0x5a9/0x830
[  465.696458]  ? getname_flags+0x10/0x190
[  465.698556]  SyS_execve+0x31/0x40
[  465.700635]  do_syscall_64+0x5b/0xc0
[  465.702679]  entry_SYSCALL64_slow_path+0x25/0x25
[  465.704718] RIP: 0033:0x7f56fedfa5e7
[  465.706771] RSP: 002b:00007ffef0b187d8 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
[  465.708872] RAX: ffffffffffffffda RBX: 0000564876f05c18 RCX: 00007f56fedfa5e7
[  465.710970] RDX: 00005648775612b0 RSI: 00007ffef0b18900 RDI: 0000564876f05c18
[  465.713097] RBP: 00007f56ffb4b3c0 R08: 00005648775612b0 R09: 00005648775612b0
[  465.715220] R10: 00007f56ffb4b440 R11: 0000000000000246 R12: 0000000000000000
[  465.717356] R13: 0000000000000000 R14: 0000564876ee0800 R15: 0000000000000000
[  465.719526] zsh             D    0   537    434 0x00000000
[  465.721685] Call Trace:
[  465.723826]  __schedule+0x239/0x890
[  465.725997]  schedule+0x3d/0x90
[  465.728139]  io_schedule+0x16/0x40
[  465.730277]  generic_file_read_iter+0x344/0x9d0
[  465.732430]  ? page_cache_tree_insert+0xc0/0xc0
[  465.734636]  ext4_file_read_iter+0x3e/0x40 [ext4]
[  465.736812]  __vfs_read+0xed/0x140
[  465.738970]  vfs_read+0x8e/0x130
[  465.741131]  prepare_binprm+0x110/0x1f0
[  465.743274]  do_execveat_common.isra.36+0x5a9/0x830
[  465.745425]  ? getname_flags+0x10/0x190
[  465.747588]  SyS_execve+0x31/0x40
[  465.749722]  do_syscall_64+0x5b/0xc0
[  465.751856]  entry_SYSCALL64_slow_path+0x25/0x25
[  465.754026] RIP: 0033:0x7fa6ac9df5e7
[  465.756165] RSP: 002b:00007fff817ba4e8 EFLAGS: 00000207 ORIG_RAX: 000000000000003b
[  465.758376] RAX: ffffffffffffffda RBX: 00007fa6ad8b3950 RCX: 00007fa6ac9df5e7
[  465.760594] RDX: 00005609fbe6d290 RSI: 00007fa6ad8b3950 RDI: 00007fff817ba9e0
[  465.762771] RBP: 00007fff817ba9e0 R08: 0000000000000001 R09: 00005609fbe584c0
[  465.764947] R10: 0000000000000008 R11: 0000000000000207 R12: 00007fff817ba5a0
[  465.767129] R13: 00005609fbe6d290 R14: 00007fff817ba520 R15: 0000000000000000
[  465.769310] systemd-journal D    0   540      1 0x00000100
[  465.771504] Call Trace:
[  465.773662]  __schedule+0x239/0x890
[  465.775797]  ? bit_wait+0x60/0x60
[  465.777926]  schedule+0x3d/0x90
[  465.780079]  io_schedule+0x16/0x40
[  465.782213]  bit_wait_io+0x11/0x60
[  465.784348]  __wait_on_bit+0x31/0x90
[  465.786482]  ? __d_alloc+0x27/0x1d0
[  465.788579]  out_of_line_wait_on_bit+0x90/0xb0
[  465.790687]  ? bit_waitqueue+0x40/0x40
[  465.792820]  do_get_write_access+0x291/0x4e0 [jbd2]
[  465.794959]  jbd2_journal_get_write_access+0x3c/0x60 [jbd2]
[  465.797130]  __ext4_journal_get_write_access+0x50/0x90 [ext4]
[  465.799333]  ext4_reserve_inode_write+0x93/0xb0 [ext4]
[  465.801524]  ? ext4_dirty_inode+0x48/0x70 [ext4]
[  465.803718]  ext4_mark_inode_dirty+0x4e/0x1e0 [ext4]
[  465.805943]  ? __ext4_journal_start_sb+0x6d/0x130 [ext4]
[  465.808153]  ext4_dirty_inode+0x48/0x70 [ext4]
[  465.810308]  __mark_inode_dirty+0x296/0x420
[  465.812424]  generic_update_time+0x80/0xd0
[  465.814543]  ? current_time+0x38/0x70
[  465.816628]  touch_atime+0xab/0xd0
[  465.818719]  ext4_file_mmap+0x4d/0x70 [ext4]
[  465.820816]  mmap_region+0x399/0x610
[  465.822856]  do_mmap+0x2cf/0x420
[  465.824882]  vm_mmap_pgoff+0xcc/0x120
[  465.826932]  SyS_mmap_pgoff+0x1bc/0x230
[  465.828971]  SyS_mmap+0x1b/0x30
[  465.830993]  do_syscall_64+0x5b/0xc0
[  465.833036]  entry_SYSCALL64_slow_path+0x25/0x25
[  465.835062] RIP: 0033:0x7f24bac12e63
[  465.837078] RSP: 002b:00007ffdedd8ba88 EFLAGS: 00000246 ORIG_RAX: 0000000000000009
[  465.839180] RAX: ffffffffffffffda RBX: 000000000000001d RCX: 00007f24bac12e63
[  465.841301] RDX: 0000000000000003 RSI: 0000000000800000 RDI: 0000000000000000
[  465.843438] RBP: 0000000000000000 R08: 000000000000001d R09: 0000000000000000
[  465.845555] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
[  465.847672] R13: 0000000000800000 R14: 0000000000000001 R15: 0000000000000003
#
########## 此处手动 crash
#
[  484.713013] sysrq: SysRq : Trigger a crash
[  484.713035] BUG: unable to handle kernel NULL pointer dereference at           (null)
[  484.715263] IP: sysrq_handle_crash+0x1b/0x30
[  484.717457] PGD 0 
[  484.717459] P4D 0 
[  484.719664] 
[  484.723820] Oops: 0002 [#1] PREEMPT SMP
[  484.725908] Modules linked in: l2tp_ppp l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel pppox ppp_generic slhc fuse snd_hda_codec_hdmi uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_core joydev videodev mousedev media intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp psmouse kvm_intel arc4 kvm irqbypass ath9k snd_hda_codec_realtek ath9k_common snd_hda_codec_generic ath9k_hw ath mac80211 crct10dif_pclmul crc32_pclmul cfg80211 crc32c_intel ghash_clmulni_intel cryptd intel_cstate intel_rapl_perf acer_wmi sparse_keymap wmi_bmof rfkill snd_hda_intel snd_hda_codec wdat_wdt snd_hda_core rtsx_pci_ms snd_hwdep snd_pcm memstick r8169 pcspkr snd_timer snd mii evdev input_leds led_class ac thermal dell_smo8800 battery mei_me soundcore shpchp mei mac_hid lpc_ich i2c_i801 wmi sch_fq_codel
[  484.738426]  sg ip_tables x_tables ext4 crc16 mbcache jbd2 fscrypto hid_generic usbhid hid sr_mod sd_mod cdrom rtsx_pci_sdmmc mmc_core serio_raw atkbd libps2 ahci libahci libata ehci_pci ehci_hcd scsi_mod rtsx_pci usbcore usb_common i8042 serio intel_agp i915 video button i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm intel_gtt
[  484.747169] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.13.5-1-lily #12
[  484.750186] Hardware name: Acer Aspire E1-471G/EA40_HC, BIOS V1.29 04/10/2013
[  484.753275] task: ffffffffafe10480 task.stack: ffffffffafe00000
[  484.756371] RIP: 0010:sysrq_handle_crash+0x1b/0x30
[  484.759498] RSP: 0018:ffff881a5f203cb8 EFLAGS: 00010002
[  484.762614] RAX: 0000000000000001 RBX: 0000000000000063 RCX: 0000000000000000
[  484.765792] RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 0000000000000063
[  484.768934] RBP: ffff881a5f203cb8 R08: 0000000000000562 R09: 0000000000000004
[  484.772081] R10: ffffffffafa71e60 R11: 0000000000000001 R12: 0000000000000009
[  484.775189] R13: 0000000000000001 R14: ffffffffafeb3ea0 R15: ffff881a54835960
[  484.778314] FS:  0000000000000000(0000) GS:ffff881a5f200000(0000) knlGS:0000000000000000
[  484.781485] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  484.784637] CR2: 0000000000000000 CR3: 0000000053e09000 CR4: 00000000000406f0
[  484.787837] Call Trace:
[  484.790999]  <IRQ>
[  484.794154]  __handle_sysrq+0x88/0x140
[  484.797327]  sysrq_filter+0x8c/0x3d0
[  484.800498]  input_to_handler+0x53/0xf0
[  484.803653]  input_pass_values.part.6+0x131/0x150
[  484.806835]  input_handle_event+0xe3/0x580
[  484.809989]  input_event+0x54/0x80
[  484.813158]  atkbd_interrupt+0x4fe/0x7b0 [atkbd]
[  484.816327]  serio_interrupt+0x46/0x90 [serio]
[  484.819517]  i8042_interrupt+0x202/0x3b0 [i8042]
[  484.822687]  ? find_next_bit+0xb/0x10
[  484.825869]  __handle_irq_event_percpu+0x42/0x1c0
[  484.829044]  handle_irq_event_percpu+0x32/0x80
[  484.832228]  handle_irq_event+0x3c/0x60
[  484.835339]  handle_edge_irq+0x7f/0x1a0
[  484.838435]  handle_irq+0x1a/0x30
[  484.841487]  do_IRQ+0x46/0xd0
[  484.844549]  common_interrupt+0x89/0x89
[  484.847645] RIP: 0010:cpuidle_enter_state+0x12d/0x300
[  484.850746] RSP: 0018:ffffffffafe03dc0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffce
[  484.853867] RAX: ffff881a5f218d00 RBX: 00000070db252641 RCX: 000000000000001f
[  484.856932] RDX: 00000070db252641 RSI: 00000010d7702656 RDI: 0000000000000000
[  484.860023] RBP: ffffffffafe03e00 R08: 00000000000049ec R09: 0000000000000018
[  484.863124] R10: 0000000000005a81 R11: 0000000000003b5c R12: ffff881a5f220d20
[  484.866282] R13: 0000000000000000 R14: 0000000000000005 R15: ffffffffafeac118
[  484.869425]  </IRQ>
[  484.872466]  cpuidle_enter+0x17/0x20
[  484.875378]  call_cpuidle+0x23/0x40
[  484.878171]  do_idle+0x18f/0x1e0
[  484.880863]  cpu_startup_entry+0x73/0x80
[  484.883418]  rest_init+0xd5/0xe0
[  484.885875]  start_kernel+0x49b/0x4bc
[  484.888185]  ? early_idt_handler_array+0x120/0x120
[  484.890386]  x86_64_start_reservations+0x24/0x26
[  484.892583]  x86_64_start_kernel+0x13e/0x161
[  484.894716]  secondary_startup_64+0x9f/0x9f
[  484.896818] Code: 3b c2 af e8 d9 23 c9 ff eb d5 e8 01 20 c3 ff 90 66 66 66 66 90 55 48 89 e5 e8 32 cc c9 ff c7 05 70 28 c5 00 01 00 00 00 0f ae f8 <c6> 04 25 00 00 00 00 01 5d c3 90 66 2e 0f 1f 84 00 00 00 00 00 
[  484.901686] RIP: sysrq_handle_crash+0x1b/0x30 RSP: ffff881a5f203cb8
[  484.904128] CR2: 0000000000000000

依云 · 2017-10-19 21:42:45

这就是硬盘用不了了啊。你弄个纯内存的系统（比如 systemrescuecd 使用 docache 参数）肯定不会卡死了。

zsrkmyn · 2017-10-21 02:04:53

哎试了吧 use_blk_mq 去掉确实就正常了，感觉还是内核的 blk_mq 的问题。哪天有空仔细研究下。

Arch Linux

#1 2017-10-18 17:42:17

唤醒之后系统卡顿假死

#2 2017-10-18 18:45:30

Re: 唤醒之后系统卡顿假死

#3 2017-10-19 01:51:49

Re: 唤醒之后系统卡顿假死

#4 2017-10-19 10:54:22

Re: 唤醒之后系统卡顿假死

#5 2017-10-19 12:20:14

Re: 唤醒之后系统卡顿假死

#6 2017-10-19 12:31:08

Re: 唤醒之后系统卡顿假死

#7 2017-10-19 12:47:13

Re: 唤醒之后系统卡顿假死

#8 2017-10-19 18:58:16

Re: 唤醒之后系统卡顿假死

#9 2017-10-19 19:33:16

Re: 唤醒之后系统卡顿假死

#10 2017-10-19 21:42:45

Re: 唤醒之后系统卡顿假死

#11 2017-10-21 02:04:53

Re: 唤醒之后系统卡顿假死

页脚