Handling of abnormal playback of jvm-sandbox-repeater dubbo

It’s still the problem of drainage playback. The students who tested today reported that he did traffic playback, but several interfaces of the playback reported errors, and they were all server errors. Please contact the administrator. The expected results are inconsistent, but the actual one The logic has not been changed, so it can only be a problem of dubbo playback.

Analysis

Let’s first look at what the phenomenon of the problem looks like, as shown in the following figure:


It is expected that the result of returning a work order that does not exist is correct, but it is indeed the result of an abnormal server.


Judging from this result, the dubbo call returned null, which caused this problem. The dubbo call is actually the result of our mock, and we can see the recorded sub-call data by looking at the code

In fact, the business is only a sub-call of dubbo, so the expected result should be that this getWorkOrderDetailByUid returns the non-existing work order we need, but we see the response of this result code> and throwable are both empty, which is why null is returned.

Then we have to think again, why both response and throwable are null. This is to look at the logic of the recording.

/**
     * Handle the return event
     *
     * @param event return event
     */
    protected void doReturn(ReturnEvent event) {<!-- -->
        if (RepeatCache.isRepeatFlow(Tracer.getTraceId())) {<!-- -->
            return;
        }
        Invocation invocation = RecordCache.getInvocation(event.invokeId);
        if (invocation == null) {<!-- -->
            log.debug("no valid invocation found in return, type={},traceId={}", invokeType, Tracer.getTraceId());
            return;
        }
        invocation.setResponse(processor.assembleResponse(event));
        invocation.setEnd(System.currentTimeMillis());
        listener.onInvocation(invocation);
    }

From the above content, we will find that in the doReturn method, the result of processor.assembleResponse(event) will be obtained. Let’s see how the logic for dubbo obtains the result of.

@Override
    public Object assembleResponse(Event event) {<!-- -->
        // Assemble the response in the before event of onResponse
        if (event.type == Event.Type.RETURN) {<!-- -->
            Object appResponse = ((ReturnEvent) event).object;
            try {<!-- -->
               return MethodUtils.invokeMethod(appResponse, "getValue");
            } catch (Exception e) {<!-- -->
                // ignore
                LogUtil.error("error occurred when assemble dubbo response", e);
            }
        }
        return null;
    }

Here we need to know what type appResonse is here. In fact, it is the result of dubbo call, which is the RpcResult class.

public class RpcResult implements Result, Serializable {<!-- -->
    private static final long serialVersionUID = -6925924956850004727L;
    private Object result;
    private Throwable exception;
    private Map<String, String> attachments = new HashMap();

public RpcResult(Object result) {<!-- -->
        this.result = result;
    }

    public RpcResult(Throwable exception) {<!-- -->
        this. exception = exception;
    }
    /** @deprecated */
    @Deprecated
    public Object getResult() {<!-- -->
        return this. getValue();
    }

    /** @deprecated */
    @Deprecated
    public void setResult(Object result) {<!-- -->
        this. setValue(result);
    }

    public Object getValue() {<!-- -->
        return this.result;
    }

    public void setValue(Object value) {<!-- -->
        this.result = value;
    }

public Throwable getException() {<!-- -->
        return this. exception;
    }

    public void setException(Throwable e) {<!-- -->
        this.exception = e;
    }

...
}

We found that there is a problem here. If the service called by dubbo throws an exception, the result of result is actually null, and the value of exception can be used to know what the exception is. So the problem with the above code logic is that when the result is null, the exception may have a value. So the following changes need to be made:

@Override
    public Object assembleResponse(Event event) {<!-- -->
        // Assemble the response in the before event of onResponse
        if (event.type == Event.Type.RETURN) {<!-- -->
            Object appResponse = ((ReturnEvent) event).object;
            try {<!-- -->
                Object result = MethodUtils.invokeMethod(appResponse, "getValue");
                if (result == null) {<!-- -->
                    return MethodUtils.invokeMethod(appResponse, "getException");
                } else {<!-- -->
                    return result;
                }
            } catch (Exception e) {<!-- -->
                // ignore
                LogUtil.error("error occurred when assemble dubbo response", e);
            }
        }
        return null;
    }

Try to get the value first, and then get the value of the exception if it is null. And it’s not over yet because the value obtained just now is actually directly assigned to the response of the invocation, that is, invocation.setResponse(processor.assembleResponse(event));, here we need to set the value if there is an exception Assignment to throwable is correct.

@Override
protected void doReturn(ReturnEvent event) {<!-- -->
    if (RepeatCache.isRepeatFlow(Tracer.getTraceId())) {<!-- -->
        return;
    }
    Invocation invocation = RecordCache.getInvocation(event.invokeId);
    if (invocation == null) {<!-- -->
        log.debug("no valid invocation found in return, type={},traceId={}", invokeType, Tracer.getTraceId());
        return;
    }
    Boolean hasException = ((DubboConsumerInvocationProcessor)processor).isHasException(event);
    if (hasException) {<!-- -->
        invocation.setThrowable((Throwable) processor.assembleResponse(event));

        invocation.setResponse(buildExceptionResponse(invocation.getThrowable()));

    } else {<!-- -->
        invocation.setResponse(processor.assembleResponse(event));
    }
    invocation.setEnd(System.currentTimeMillis());
    listener.onInvocation(invocation);
}

Here we also made a small optimization, because once dubbo throws an exception, the response will have no value, which leads to null in the place where the result is compared, and there is no way to see that it is an exception, so we convert the exception Then assign to resone. That is

private Map<String, String> buildExceptionResponse(Throwable e) {<!-- -->
    Map<String, String> map = new HashMap<String, String>();
    String clzName = e. getClass(). getName();
    int index = clzName. lastIndexOf(".");
    map.put("exception", clzName.substring(index + 1));
    map. put("message", e. getMessage());
    return map;
}

Simplify the exception content, store it in the map and return it.

Is that problem solved? No, all the logic here is to solve the problem of recording traffic. As for playback, is this logic correct?

Let’s look at the logic of dubbo playback,

@Override
    public Object assembleMockResponse(BeforeEvent event, Invocation invocation) {<!-- -->
        try {<!-- -->
            Object dubboInvocation = event.argumentArray[1];
            Object response = invocation. getResponse();

            Class<?> aClass = event.javaClassLoader.loadClass("com.alibaba.dubbo.rpc.RpcResult");
            // Call AsyncRpcResult#newDefaultAsyncResult to return
            Constructor constructor=aClass.getDeclaredConstructor(Object.class);
            return constructor. newInstance(response);

        } catch (Exception e) {<!-- -->
            LogUtil.error("error occurred when assemble dubbo mock response", e);
            return null;
        }
    }

The above code is the logic of return value in dubbo mock, but according to the previous explanation, we will find that there is a problem here, that is, the constructor of RpcResult can have object parameters and throwable parameters, and this The local mock has always been simulating the return of normal results, without the logic of business throwing exceptions, so this place must also be modified.

The modified code is as follows:

@Override
    public Object assembleMockResponse(BeforeEvent event, Invocation invocation) {<!-- -->
        try {<!-- -->
            Object dubboInvocation = event.argumentArray[1];
            Class<?> aClass = event.javaClassLoader.loadClass("com.alibaba.dubbo.rpc.RpcResult");
            // Call AsyncRpcResult#newDefaultAsyncResult to return
            Constructorconstructor = null;
            if (invocation. getThrowable() != null) {<!-- -->
                constructor = aClass.getDeclaredConstructor(Throwable.class);
                return constructor. newInstance(invocation. getThrowable());
            } else {<!-- -->
                constructor = aClass.getDeclaredConstructor(Object.class);
                Object response = invocation. getResponse();
                return constructor. newInstance(response);
            }

        } catch (Exception e) {<!-- -->
            LogUtil.error("error occurred when assemble dubbo mock response", e);
            return null;
        }
    }

If invocation has throwable, then it is necessary to construct an RpcResult object with exception.

PS: In fact, there is another important “small problem” here, that is, some business exceptions that occur during our dubbo call process cannot go to the throw exception defined by sanbod, which is the place of the following code.

public void doMock(BeforeEvent event, Boolean entrance, InvokeType type) throws ProcessControlException {<!-- -->
    /*
     * Get playback context
     */
    RepeatContext context = RepeatCache.getRepeatContext(Tracer.getTraceId());
    /*
     * mock execution conditions
     */
    if (!skipMock(event, entrance, context) & amp; & amp; context != null & amp; & amp; context.getMeta().isMock()) {<!-- -->
        try {<!-- -->
            /*
             * Build a mock request
             */
            final MockRequest request = MockRequest. builder()
                    .argumentArray(this.assembleRequest(event))
                    .event(event)
                    .identity(this.assembleIdentity(event))
                    .meta(context.getMeta())
                    .recordModel(context.getRecordModel())
                    .traceId(context.getTraceId())
                    .type(type)
                    .repeatId(context.getMeta().getRepeatId())
                    .index(SequenceGenerator.generate(context.getTraceId()))
                    .build();
            /*
             * Execute the mock action
             */
            final MockResponse mr = StrategyProvider.instance().provide(context.getMeta().getStrategyType()).execute(request);
            /*
             * Processing strategy recommendation results
             */
            switch (mr. action) {<!-- -->
                case SKIP_IMMEDIATELY:
                    break;
                case THROWS_IMMEDIATELY:
                    ProcessControlException.throwThrowsImmediately(mr.throwable);
                    break;
                case RETURN_IMMEDIATELY:
                    ProcessControlException.throwReturnImmediately(assembleMockResponse(event, mr.invocation));
                    break;
                default:
                    ProcessControlException.throwThrowsImmediately(new RepeatException("invalid action"));
                    break;
            }
        } catch (ProcessControlException pce) {<!-- -->
            throw pce;
        } catch (Throwable throwable) {<!-- -->
            ProcessControlException.throwThrowsImmediately(new RepeatException("unexpected code snippet here.", throwable));
        }
    }
}

We can find that the mock strategy here will follow the specific strategy logic based on mr.action, (but we should pay attention to some exceptions in dubbo sub-calls, we still need to use RETURN_IMMEDIATELY This is a comparison Crucially) how did mr’s action come from? We need to look at the logic of execute

response = MockResponse. builder()
                        .action(invocation.getThrowable() == null ? Action.RETURN_IMMEDIATELY : Action.THROWS_IMMEDIATELY)
                        .throwable(invocation. getThrowable())
                        .invocation(invocation)
                        .build();

Here we have not intercepted a lot of code, we can see that the action here is processed according to whether there is a throwable, so once our logic goes like this, it will enter the THROWS_IMMEDIATELY In fact, the mock failed, so we need to change the code here.

response = MockResponse. builder()
                        .action(invocation.getThrowable() == null || invocation.getType().name().equals(InvokeType.ALIBB_DUBBO.name()) ? Action.RETURN_IMMEDIATELY : Action.THROWS_IMMEDIATELY)
                        .throwable(invocation. getThrowable())
                        .invocation(invocation)
                        .build();

If it is called by dubbo, there is no need to throw an exception.

Summary

There is still a big difference between the mock of dubbo and http, etc., so there are still many problems to be solved in the mock of this piece.