问题版本
mongodb-v3.4.4
问题描述
mongos两天死了两次，死前遗言只有日志：
2017-11-01T11:25:27.135+0800 F - [NetworkInterfaceASIO-TaskExecutorPool-1-0] Failed to mlock: Cannot allocate memory
2017-11-01T11:25:27.140+0800 I - [NetworkInterfaceASIO-TaskExecutorPool-1-0] Fatal Assertion 28832 at src/mongo/base/secure_allocator.cpp 246
2017-11-01T11:25:27.140+0800 I - [NetworkInterfaceASIO-TaskExecutorPool-1-0]***aborting after fassert() failure2017-11-01T11:25:27.182+0800 I NETWORK [thread2] connection accepted from 100.97.227.0:37132 #282914 (1156 connections now open)
2017-11-01T11:25:27.191+0800 F - [NetworkInterfaceASIO-TaskExecutorPool-1-0] Got signal: 6 (Aborted). 0x55db5fab1241 0x55db5fab0339 0x55db5fab081d 0x7f848d578390 0x7f848d1d3428 0x7f848d1d502a 0x55db5f2e90e7 0x55db5f3452cd 0x55db5f407733 0x55db5f4039ce 0x55db5f40538c 0x55db5f3da9c7 0x55db5f3fc16f 0x55db5f3fedaf 0x55db5f7dba02 0x55db5f7dc1be 0x55db5f7ddf64 0x55db5f7a69d4 0x55db5f7b9498 0x55db5f7b9ae2 0x55db5fb8edd4 0x55db5fb8efd1 0x55db5f7cf91f 0x55db5fe8e120 0x7f848d56e6ba 0x7f848d2a482d
关键点：Failed to mlock: Cannot allocate memory和Got signal: 6 (Aborted).
排查
mongodb的jira上有两个issue跟这个一模一样：
分析
SaslSCRAMSHA1ClientConversations have a SCRAMSecrets which they ‘ll pull out of the cache. SCRAMSecrets allocate secure storage in their default constructor, so they may be populated. Instead, SaslSCRAMSHA1ClientConversation and the cache should store shared_ptrs to SCRAMSecret.
大致意思就是说SaslSCRAMSHA1ClientConversation这玩意儿每次初始化对象都会申请一片安全的存储，这个会填充memlock区域，正确的做法应该是共享这片区域。
处理
按照SERVER-29086的意思，可以调整memlock大小，来缓解这个问题。
按照SERVER-28997的意思，这是3.4.4版本的bug，所以可以升级到bug-fixed版本3.4.6，这是最终的解决方法。
注意：要多多关注各种issue，jira，邮件列表。
